It sounds like to need to do the calibration on the individual game you're playing-- what game(s) have you tried?
Some games are more finicky than others. Arcade games had several types of lightgun systems, which with the AimTrak, we're trying to emulate using one system to cover them all.
Once you find a game that obviously responds to your tweaking, the rest will be much easier to figure out. Still, there will likely be some gun games that still just won't work; for example, I'm using MAME v112, and off-screen reload doesn't work for me in Lethal Enforcers. Works in other games, just not in LE. Beast Busters is another one-- I can't get the gun calibration in BB to save properly, so my shots on screen are nowhere near the MAME gun cursor.
The two things (beyond what you said you already did) that I have to do on most games to get the gun games to work:
1) Figure out how that individual game handles gun calibration. It's usually through the test menu (F2), but it differs from game to game. For example, one I set up yesterday accessed the test menu via F2, but to access the gun calibration portion I had to hold the service switch (9) while I pressed F2. The info in the MAWS database can often help with this.
2) Tweak the mouse sensitivity settings in the Tab menu (analog controls). I'm using an ArcadeVGA hooked up to an arcade monitor, and I've noticed that when I run games at their native resolution-- 320x240 as opposed to the desktop at 640x480, say-- the MAME gun cursor is restricted to a small area at the center of the screen, and I have to increase the X and Y axes sensitivity until the MAME cursor matches my gun sight. If I run a game using hardware stretching or Direct3D at 640x480, the cursor usually starts out with a much larger range than with the games running in smaller resolutions.
But if you have a properly aligned MAME cursor, and your shots aren't going to the cursor, it's almost certain that you need to activate that particular game's gun calibration.