Yes, remove the black things with the green dots.
No, leave the clear things there.
Generally, the middle of the three holes you uncover when you pull the black things will have the +5 power. You only have to hook up one.
Ground will be whichever trace connects to all three buttons. Looks like there's an empty hole for ground where the middle mouse button would be, so that's easy. If you want the mouse buttons, you remove the tiny microswitches and the button inputs are the traces that don't connect to ground.
On a Happ TB, red is +5, black is ground, yellow and green are one axis, and blue and purple are the other. As long as you get red and black right, you can afford to experiment with the others.
*edit- You're using a Betson, not a Happ. The connections are the same, except that the purple wire is brown instead.