Pics would probably help tailor suggestions to the specific panel in question.
Ok, I've got some pics and hopefully they will help.
This is the panel when it is locked down:

Here are the right and left sides:


The rubber rings were an attempt to lift the panel up a little so the clamp would be more snug. It worked but it has gotten loose again. The main problem is that the panel slides towards the player when pulling down on a joystick.
Here is how the clamp attaches to the panel:

I can take more if necessary...
~telengard