Make sure you have the little lock washer in there, and use a wrench to hold the shaft while you tighten it.
I don't have the ball tops, and I can't remember if they came with the bat tops or the joystick, but I have a small lock washer in there that snugs up against the bottom lip on the shaft of the joystick. Snugged up, it holds just fine.
If I didn't have them, I'd just use a little lok-tite purple and snug it up with a wrench. They'll come off easily but that would keep them from turning. If the purple doesn't hold it, I'd use a little blue.