You are going USB to connect your panel to your cab?
IMO a swappable approach is best. Make three panels. Each panel can be the width of your cabinet, no need for it to jut out.
1) 2 joysticks with 7 buttons each, or 8 if you like playstation emulation. Build this panel first.
    
            x  x  x                x  x  x
   O       x  x  x        O     x  x  x
         x  x                   x  x
2) a dedicated 4-way joystick, three buttons, and a trackball, for your classic 4-way games, Centipede, Missile Command, etc.
                           
     O     x  x  x     TB
3) a dedicated Defender joystick, 6 buttons in a Stargate-like layout, a spinner, and a flght stick, for Tron, Defender, Star Trek, Tempest, and so on.
                                    x   x     
            O         SP     x          x      FS
                x            x                
 
But that's my opinion, one of many.