Wow, thanks for the quick response! Duh, I should have posted the images, first but here they are:

Underside of the panel. As you can see I have removed all the buttons and wiring. I'm thinking about replacing this piece of wood as well so I can design my own stick and button layout.

Closeup of the left stick, which I just unscrewed the 4 screws holding it to the wood.

Closeup of the right stick, which I went a little further and took off all the microswitches and the "bottom" of the joystick.
Any suggestions?