I just finished my own 3-sided portable cocktail and faced the exact same issues. I ended up using an IPAC4 and wired the vertical end positions as players 3 and 4, with 1 and 2 being the horizontal side-by-side.
I LOVE this configuration, although I will admit that there is some input programming necessary for some games. I use a game favorites list, and as I add something to it, I try every configuration for the best orientation and control setup which does lead to some remapping of the inputs.
For me, every player has a start button and a coin drop, so on vertical games, I usually double-map P1COIN and P3COIN (the vertical player 1) so you can hit the P1 coin or P3 coin for a credit, depending on where you are sitting. I do the same thing with the start buttons.
Cocktail DIP switches are in a lot of the vertical games, but I found a few that don't work properly. For instance, DigDug seems to put the player2 sprite in the wrong place when flipped for some reason. So, for these, I keep it rotated and leave the DIP switch to UPRIGHT.
I am using the MA front end in horizontal mode only. It's a cocktail, so if a person can't move 2 feet or turn their head to select games, well, that's just silly.
I skipped the trackball for space, but I did drop in two TurboTwist2 spinners! This will let my friends and I play Blasteroids side-by-side, Off The Wall (sente) and a couple of others. Although my spinners are on the horizontal side, my panels kinda wrap around the screen, so it's easy enough to just use the left spinner at the vertical side for games like Tempest. By the same token, two-player SmashTV works like a champ by sitting horizontally and grabbing both sticks!
For me, the programming time was worth the effort to get vertical games running vertically, cocktail games flipping if they can, and you just can't beat 4-player Gauntlet action!
[youtube]http://www.youtube.com/watch?v=3L7L7BNoo9A[/youtube]