No you'd need it all switched.
The controls could be switches seperately from the A/V using seperate control an A/V switches. You can buy A/V switches straight out of the box anywhere. That would reduce the number of lines required for switching. It's not as clean, though. Besides, if you're already switching 30+ cables, what's a few more?
I haven't heard anything bad about them. Have you found a source for connectors?