I have one of these now (and a nifty small plastic case that cost me £3+shipping) but I havent started experimenting with it. My biggest gripe with the RPi is that it doesnt have a VGA port, hence you cant use an old 4:3 LCD which you can find cheaply and in abundance now (I actually found two free ones in a scrap heap today, hope one of the works). So you need something with a DVI port then since there are HDMI-DVI adapters from what I hear, and most DVI port screens are either bigger ones or widescreen ones (hard pressed to find e.g. a 15" with DVI). The same with composite, you have to find a TV then (CRT or LCD). Ofc going CRT you can try to get the real arcade feeling if that is your thing.
Shea Silverman has a nice blog about the RPi and getting emulation working on it including his Mame setup. Will be trying this out once I get my RPi up and running although I am at loss to find a screen I can use for a DonkeyKong build (15" screen).