What type of toggle switch, 2way? End panels to one input and the 3rd panel to the other input?
Use a SPDT (or DPDT) switch. The SPDT will have 3 terminals. The center is common, the switch will switch between the two outer ones. (Assuming that you orientate the switch so that it switches up and down), check the terminals to see which is which before wiring, it should be that when the switch is up, the bottom term. will connect to center. Connect ground to center. Connect the up terminal to all the grounds of the switches and joys of the end panels. Connect the down terminal to all the grounds of the switches and joys of the side panel. When switch is up, side will work, when switch is down, the end panels will work.
A DPDT will have 6 terms. It works the same except that it switches two sets of contacts rather than just the one that the SPDT.
With the u360's, perhaps you could have your front end program a custom map into the unused (per game) sticks that contains all center cells. Say your side panel has #1 and #2, while the ends have #3 and #4. When playing a horz game, have your FE program 1 & 2 to (say) 8-way, while programming #3 & #4 to all center. Not sure if FE's can program 4 at a time...
Rick