I have a 3 buttons and a trackball connected to a Opti-Wiz. I have a fire button and joystick connected to a GP-Wiz. For joystick games I want to sometimes use the 3 buttons from the one encoder. When playing trackball games, I may want to use the one fire button from the other encoder. But if I touch the joystick at all, my mouse starts jittering and veering off, as if the GP-Wiz is fighting input for the Opti-Wiz. If I wait long enough, the opti-wiz regains control and things are back on track, quite annoying though.
If I disable joy (ie. no "-joy") then my fire button doesn't work. If I disable trackball, then my 3 other buttons don't work.

Both your trackball and joystick are mapped to the directions at the same time. For games you want to use the TB, map only the trackball to the inputs "... analog" and make sure the joystick is not mapped to ... analog, ... DEC, and ... INC. For games you want to use the joystick, unmap the TB like you did to the joystick above.
Mame FAQ is better at describing how than I am.
(You might want to make two ctrlr files, one for TB the other for stick, and enable one or the other game b game, rather than remapping each game individually.)
Easyemu is a good starting point.