OK. I think i see the problem. If you look closely, you will see two rings at the bottom of the joystick shaft for the e-clip. The one farthest from the joystick parts is used if you mount it on a wood panel (it's currently empty). The one closest to the joystick (look closely, it's there with the e-clip on it) is if you mount it on a metal panel. You have a metal panel, so the e-clip is lower.
To remove it, push the joystick from the handle towards the bottom of the shaft. While doing this, push the activator (the square plastic piece) towards the top of the joystick. It takes a little finesse to do this with one hand. This will allow the e-clip to be exposed enough for you to grab it with a set of needle nose pliers and pull it out. Once it's out, simply pull the joystick out. Or if it's a little rusty, you may need to carefully tap the bottom of the joystick shaft to coax it out. Be careful not to hit it too hard. The control panel may bend if you just whack it full force.
Here's a pic from CoinopExpress.com for your reference showing the e-clip (this one is currently installed for a wood panel on the lower groove (see below)
Good luck!
DeLuSioNaL29