The Mac Mini can run Windows just like any other PC. It is dead silent, tiny, and looks better than all of the HTPC cases I've seen. The TV out on them is fine, much better than my old mini-itx systems, and they have DVI built in, so if you have a newer TV you can probably use that and have a perfect picture. And yes, they have USB.
If you want to avoid Windows, there's a bunch of OS X based emulators too, though Pik4chu is right in that there aren't any pinball ones (at least that I've seen). There don't seem to be many front-ends though.
The big risk with getting a Mac Mini is that you might do what I did - scrap your intended project and start using the Mac as your main PC.
