SirP- you wanted to get the mode selection without 8 buttons on the panel. I figure it can be done with fewer, here's how:
Randy says the mode button is a standalone, with no other function, so that's one right there. Of course, since it's not used during play, it could be a non-arcade button, so you could mount a really small button anywhere to do this job, even conceal it inside the coin door or whatever. (As far as that goes, you could conceal all 8, but let's assume you don't want to do that.) That leaves 7, one for each of the modes.
When pushed, the mode select button essentially shifts button inputs 1-7 from their standard function to "Mode Select". If you've got coin1, coin2, start1 and start2 on your machine, there's 4 already Got 4players, then you're covered right there. If not, that still only leaves 3 "fire" buttons to go on the panel. If you've got admin buttons, mouse buttons, etc.- you can use those.
Even if your other buttons are wired to another encoder (Ipac, Keywiz) that has their coin, start, and player buttons wired there can use this trick too, they just have to do some more wiring. Double wire your buttons to both encoders, but wire a toggle switch in the circuits so they're only live to one encoder at a time. Wire the toggle to a relay off of your mode-select button and you're good to go. Hmm, there ought to be a way to skip the relay- maybe a DPDT in the mode select button? 3P3T?