MAME will do this if you have a nVidia 6xxx card, Windows XP 32-bit and drivers newer than about 4 years old.
It appears to be a nVidia bug, MAME isn't really doing anything too unusual but blows up, as you say, in a very non-graceful way on a draw call to the nVidia drivers.
use (much) older drivers.