Don't blame me, I didn't draw it. Just took it from another site.

Personally, I just use the bottom set of buttons (my panel has a similar layout to a Hotrod SE). Neo-Geo type is great for 4-button games.
I'll see if changing some button assignments will help. Everything else plays just fine, at full speed.
Edit: Found out it's not the software at all. I have an intermittent short somewhere in my CP wiring. I never found it in my other software because it's the 7th button (the least used in most of the games I play) and because I usually play as Player 1 (problem is on player 2). Shouldn't be hard at all to fix (famous last words...), but I must leave that for another day.
Thanks for your help! Knowing others have it working correctly pushed me to find the fault with mine.
Edit 2: Turns out, the P2 SW7 on my IPAC was bad. My control panel is wired using 15-pin connectors, and I use that same pinout with my desktop arcade panel and IPAC rig (great for taking games on the go!). Hooking up just the 15-pin cable to the P2 connector on my cab, but before I hooked up my joystick, and SW7 was sticking on. Most likely I burned out a pull-down (or pull-up) resistor when I hooked up the the IPAC and Naomi interface wires together (only buttons 1-4 were supported on the Jamma harness, and I thought I read that I can just hook up the other action buttons from different boards together...) An easy fix, though. Just moved the wire to a different terminal on the I-PAC (it's a 4 player model in a 2 player cabinet. Plenty of open spaces.) and remapped the buttons in the IPAC config software. Now everything works perfectly.