I'm going to use a Raspberry PI running pimame.
(Thinking Street Fighter, Tekken, Mortal Combat)
Not sane. Prepare to be disappointed if you expect those to run full speed on a Pi.
Hell, most of the people on here running Pentium 4s can't run Tekken in MAME and have to use zinc emulator.
The control panel you linked to is actually a decent example.
Notice how the joysticks are all parallel to the screen regardless of how the buttons are angled.

On the number of buttons:
The most buttons used by a 4 player game in MAME is four and it's only few Dungeons and Dragons games. The rest use 3.
Guilty Gear Isuka uses 5 buttons, but it is only playable on Demul emulator and requires much more horsepower than you're talking about using.
So for arcade games anyway, you're safe with 4 buttons for players 3 and 4.
Ignoring games that only had buttons and no joystick, the maximum number of buttons in a two player game is 6.
As was said earlier, a lot of people add a 7th so they can keep the Neo-Geo four-in-a-row and have the run button in Mortal Kombat 3 closer to it's original position. I like having the Neo Geo button colors because during the attract mode, NeoGeo games give instructions about which button does what.
The 8 button panels that turn up are usually built by younger people who are modeling them after fight sticks currently on the market.
The two extra buttons on the SFIV sticks just post combinations of the other buttons. The real arcade version only has 6.
As far as console emulation goes, if you're emulating something that requires more than 6 buttons, it also requires analog sticks IMO.