I don't know about plain MESS but MAMEplus has that cocktail mode for the consoles that it supports.
There are a handful of PS2-era shooter games that can run in what they called "TATE" mode, because "rotated" wasn't snazzy enough I guess. They are mostly duplicates of arcade games but some of them have different modes.
There was an Apple II version of Galaxy Wars that rotated the monitor, but again it's just an arcade port.
Some of the Touhou shooters can run in vertical mode if you use a patch.