I think what he was saying...
If you choose to switch the grounds, then you've NOT switched the control wiring.
Which means, if I understand your concept, that you will have the "Player 1 - Left" from the IPAC tied to the same switch contact as the "Player 1 - Left" from the game controller.
It might work, but I'd be scared to try it. The IPAC supplies a voltage, probably 5v (never checked mine) to one side of your switches. When you activate the switch, it gets pulled to ground, and tells the microprocessor in the IPAC "Hey, this just happenned."
So far so good... But the game pad ALSO supplies a voltage, which may or may not be compatible with the IPAC. Since you've put both "Player 1 - Lefts" under the same lug, they are tied together, and you introduce the chance for some kind of problems...