It might be a better idea to router DOWN into the wood from above for very short joysticks. That way, you only have to lose about 1/4" of wood and therefore retain the strength.
I have a 4-way that's a MsPacman/Galaga replacement made for metal CPs, and it works great this way. First, I cut a hole that's the size of the body of the joystick (about 2.5" square) all the way through the wood. Then I routered about 1/4" above and below and 1/2" on the sides around the hole for the lip of the joystick plate. Last, I freehand routered out some 1/2" holes to allow the tabs on the microswitches to fit thru. Cover it all up with some formica or lexan and printed artwork drill holes for your joysticks and buttons, and you'd never know the difference.