I had a single-interface cat-6 cabled swappable set up on my first rig .It was OK, but you never wanted to swap, it was too many cables, etc. I now just throw down and put whatever interface I will need on the control panel itself, and then have 1 or 2 usb cables connected. Much easier to deal with.
An optiwiz costs like 15 bucks. You can connect a trackball, 3 buttons, and a spinner to it.
A Xin Mo 2 joy 22 button joystick interface - $25
The LED controllers are what will break the bank. Esp if you're doing RGB.
Spinner/Trackball CP: 2 Spinners, 1 trackball, 6 buttons (3 on each far side) -you'll need 2 interfaces - $30, I would just do 1 spinner here(arkanoid), then you only need one interface....
Fighter: 2 joysticks , 6 buttons each side, 1 & 2 player - 1 interface - $25
Classic 'Multi': 2 Joysticks, Spinner, 7 buttons (2 on left, 5 on right, close to Multi-Williams Layout) - if you use the u360's here, you just need 1 addtl optiwiz - $15
Personally, I'd skip the saitek. They never look 'arcade', and flight sticks get very limited use in most set-ups. But thats a real personal decision. For all
I know, Afterburner is your favorite game....
Thats not a bad outlay; $70 in interfaces for 3 panels.