Are you looking for a permanent fix?
If so, instead of drilling a round hole on the CP for the joystick to go through, cut a diamond. You would need to go small enough so that if you push the stick all the way left that it just touches the point of the diamond, same with up/down/right
I don't know how long it would take for normal use to 'erode' your cutout- obviously a metal CP would last almost forever, mdf would be about the softest.
Using a wood/plexi or lexan layered CP, you could cut the wood hole to normal 1-1/8" size and cut the smaller diamond just through the plexi/lexan
Visible wear would be covered by the plastic disk, also