There have been many discussions on this idea and I (as well as others have created casual portable devices using flash drives. Bear in mind though that the wisdom of using a removable/external drive for regular gaming use is questionable though as the constant read/write cycles made by emulators creating/loading/saving config and save files may slow down performance or cause errors over time.
I use Compact Flash in my sub-mini cabinets as I can boot to them using old Thinkpads. I have one that gets played with regularly and there is no sign of any errors or anything being described by Turnarcades. You can have it run DSL and have it load into memory (128mb) and pull the roms with Xmame off the flashdrive, assuming that the pen drive is boot-able, and the machine can boot from it, etc.
I have a new build runinng MicroXP, and Mame32 with Daphne, and it works just fine. The newer pen drives have more capacity and durability than pen of drives 2 years ago. Heck we are now looking at 32gb SSD drives, so it seems to be a better alternative than running off a CDROM. I wish it was the same price though....
There is a lot of trial and error, and experimenting. I went through 5 pen drives of several makes before I found one that could be read by the laptop at boot. CF cards are more durable and have better capacities, and you can substitute the laptop drive very easily, if this is the direction you are looking at going.