you can also use hwstretch, and then don't let mame change resolution OR bpp. then unless your using an imbedded intel video chipset (or was that ibm?) you should be fine. Of course, if you video card can't handle hwstretch (anything TNT or above should be fine) then it will run like crap. But you should see everything ok
but if you don't have hwstrech or d3d stretching, then you better have a near exact resolution match. Because the game will output in whatever it thinks is closest. And that might not really be what you think it is. (ie, it might double all pixels (very fast without hwstretching) and that might be 640x520 or something like that.
Also, make sure that your monitor isn't stretched out. you might be able to change the settings of your video card to handle it.