I'll chime in first with my approach. I went with multiple panels, as there was no way that I could cram all of the stuff I wanted onto one. The main problem is the diversity of control schemes used for vector games combined with the small number of games and the overall very high quality of them. Who would build a vectormame and not want to play Star Wars? Well, then you need a yoke or analog joystick, at the very least. Or Tempest, Major Havoc, Star Wars, Tac Scan, Omega Race, and Zektor? Then you need a spinner. Quantum-trackball. Lunar Lander-thruster. When you think about it, about the only scheme that wasn't used on a vector game was the "eight-way JS with 4-6 buttons on the right" pattern that every other mame machine on the planet has. In the end, just too many compromises with a single panel, at least for me. With swappables, I can also add more panels as time and money permit.
I haven't really given the B/W color issue mush thought, although it's clear that there's much less diversity in the B/W games. A lot of them (most?) are playable with the ol' "Asteroids" layout.
As for which are essential games, there are only a few real dogs, with the craptastic Barrier standing out... I'm also not really into most of the other Cinematronics games (Starhawk, Ripoff, Solar Quest, Star Castle, etc.), but that is just one guy's opinion. Heresy, I know, but I'm pretty much an Atari guy...