The only problem with these really tiny PCs is that you won't be able to play some newer games. I don't know exactly what the limits are, but the reason is because the video chipsets are not the most robust, and they use shared memory (the video memory is shared with the PC memory & vice-versa). Where the limits are, I don't know.
Personally I would go for a mini-itx or something similar with a single PCI (or agp) expansion slot. You can throw a cheap GeForce 5200 PCI or something like that in there and not have to worry about those problems, or find an older ATI card w/ TV out (like a 9250 based or maybe even a 9600-based PCI card and you get the TV out as well.
Also the speed of the CPU, IIRC is a little bit confusing. For example the VIA EPIA MBs with the 1GHz cpu will not have the benchmarks of a celeron 1GHz, or the equivalent AMD. It's slower. More like the equivalent of a Pentium *I* 1 GHz CPU (which obviously doesn't exist, but it gives you an idea) rather than a P4, etc.
On the plus side with many of these cooling is not an issue with them like with a regular PC CPU. They dont run as hot and are designed to be enclosed in small spaces. Usually the fan that is in those tiny cases is plenty.
They tend to be a bit pricey retail, but good deals can be found on eBay for these.
As someone else pointed out, go to mini-itx.com and look at all of the projects, and read up on the different boards/cpus etc. There are some REALLY cool projects up there!
Brain21