Modular, Modular all the way.
I bought a Slikstick Frankenpanel, while it was convient, it was also the size of Kentucky and completely unusable without a 50" screen.
I then tried building swappable panels, which worked ok except I need like 8 panels and they took up an incredible amount of space.
So then I build a modular panel, bought a cheap bookshelf, and have any layout I could possibly want with no more complexity in use than a swappable, and significantly less space requirement. I'll never go back. Heck, just today I saw the 2-way joystick, and I can fit it into my control scheme with a very minimal amount of effort. My only wish is that there was a place to buy clean wiring harnesses for the controls ending in ethernet ports, it took me longer to work out the wiring than it did to design and build the panels and box, due entirely to all the wiring fuss.
Honestly, if you're looking at Swappable panels, you really should consider modular. Outside of the artwork and the slight edges between them, there's really not much difference in design or use.
Though I admit, the artwork issue is definitely a valid complaint, I understand it's important to many.