Sounds like your button keeps the contact closed. What Witchboard meant by "momentary switch" was "does the switch open when you stop pushing it." A standard pushbutton has a momentary switch. A lightswitch in your home is not a momentary switch.
If the switch stays closed when you stop pushing it, you should use a different switch. MAME expects one push to pause and one push to play.
To kind of expand on what jcoleman said, I have this on my site:
Maintained switch: When you click it on it STAYS on, when you click it off it STAYS off. Think of a regular wall-mount light switch in most homes - when you turn it on by flicking the switch, the switch physically stays in the "on" position until you flick it back off.
Momentary switch: There is no "on" or "off" position. You push it in, it send its signal, and it pops right back out when you let go of it. An easy example would be your television remote control: you push the same "power" button to turn your TV on and off, but you can't physically look at the switch and tell if it's on or off. The switch doesn't stay in while it's on and pop out when it's off. It sends its signal while the button is held down, and stops sending its signal when you let go of the button.