Ok the possibility for conflicts is high, but at the risk of over-simplifying things....
First off, forget about common ground and piggy backing it... that's out.
What you would do is piggy-back switches in a series of stand-alone loops like so:
I-Pac GND ---------->SW1 GND [switch 1] SW1 NC --------->SW2 GND [switch 2] SW2 NC------->I-Pac P1 B1
The above loop would only complete if both switch 1 AND switch 2 are closed.
The problems would be switches 3 and 4, which only need one switch closed... my bet is they would gum up the works.
A far easier solution is to either hack a keyboard encoder or use a teensy outputting as a keyboard. Then you can do a 2-d array and things solve themselves.
A teensy is cheaper than an I-pac and has enough I/O pins to easily handle the shifter and any buttons you would have on the control panel, so if it were me, that's what I would use.