Need some advice on fixing this flush mount for a trackball.

[sin360]

Ok so I have my panel pretty much finish except for the track mount. I was following the guide that Epyx made for my electric ice. I didn't feel comfortable cutting the recess out so I took it to a wood shop.  Explained to them what I wanted and even showed them pictures gave them the specs and everything. Of course people can't follow directions I go pick my top panel up and the lip off the ball doesn't come all the way through to sit flush and the hole is a little too big. It's not sitting square on the hole either. I said F*ck it I'll fix it myself. So I need to make the outline just a hair bigger so I have enough room to move the trackball around to center the lip on the hole and recess it a little more.  What would be the best way about doing this? I was just going to make some sanding blocks and go to town and then chisel the outline a little bigger.

If anyone has any advice it would be greatly appreciated. I forgot I was thinking of using some black silicone to fill in the little bit of gap in the center hole.

So here is the guide that I got the idea from.
http://forum.arcadecontrols.com/index.php?topic=100903.0

[sin360]

Here is the recess cut out on my panel.