Bah . . . just router out your recesses. I've routered loads of plywood and never had any trouble at all. And I don't have nearly the patience to do multiple passes.

Seriously, I'm sure there's truth to the warnings here, but 9.9 times out of 10 the plywood will behave just like solid wood when you put a router to it. And if you're working on the bottom, it doesn't even matter if you happen to have the .1 that goofs up a bit. It's not like your piece is going to crack in half or something. You might just end up with a little void or something irrelevant like that.
For what it's worth I only have three panels. All are plywood. All are recessed on bottom for the joysticks and two are recessed on top to make the trackball mounting plate and the Ms. Packman/Galaga reunion stick mounting plate flush. It is seriously nothing to worry about at all. Routers play perfectly well with plywood.