You can actually buy special drill bits for using in routers. I've seem them advertised. Unfortunately they cost a lot more than standard drill bits.
Another thing that occurs to me is that most routers have a variable speed motor. Maybe some models can be set to spin slow enough to handle standard drill bits. Just a thought, I'm not a woodworking expert.
The router bits you are talking about are plunge cutting bits. Generally they aren't for drilling holes they are to plunge your router into the work and cut from there.
They do not make a router that will spin slow enough to handle standard drill bits. Please for you own safety and the safety of others around you never chuck anything in your router that isn't designed to go in your router. Read my post above about the drum sander bit. To expand on that story a little he actually tried a smaller drum sander bit that seemed to work well so he decided to use the next size up. After bouncing around the room a few times it stuck itself into the wood wall. I've never had a router bit come loose but I've had boards kick out of a tablesaw or explode on a CMS sending shards in every direction so I don't do things that I know are unsafe. I like to be able to see and count to 10 on my fingers so I respect my tools and take reasonable safety precautions.
With a circle template, template guides and a pluge cutting bit you could cut circles but this seems like a lot of extra work with no significant gain. Arcade buttons holes don't need to be perfect since your going to over lap the edge with the button anyway.
Forstner bits in a drill press are best, a forstner bit in a drill guide would be good, and you can get acceptable results hand holding the drill with a forstner bit. Can't afford the $8 bit you can go get a whole set of spade bits for about $4 and still get acceptable results.
Buttons use a 1 1/8" hole and most joysticks use the same size but some people drill them 1 1/4", you might have to go a little bigger with joysticks with a long throw, a thick panel (more than 3/4"), or if don't get the joystick centered in the hole.