Well DYNAGOD, if you connect is like a TV, then it won't look much better than a TV. (Depending on how much de-interlacing etc the HDTV has.) The trick is to understand what the HDTV is best at and how to take advantage of it.
Okay, one big assumption: we're talking about a true HDTV with 720p or 1080i resolution, not some widescreen TV which has only 480 lines.
HDTV has two 16x9 hi-res modes: 1280x720@60fps (or 720p) and 1920x1080@30fps interlaces (or 1080i). Not all HDTVs can display both modes, but rather they use scaling to display the other. Most HDTVs will also upscale other resolutions (e.g. 480i or 480p) to a higher, preferred or native, resolution for display.
Now, if you connect your PC to the HDTV via TV-out then you are, in effect, converting whatever resolution the game is in to 640x480i to send it to the HDTV. The HDTV then will upscale, and probably deinterlace, that to fit it's preferred display mode. Some graphics cards may support progressive output, which means avoiding deinterlacing artifacts.
Now, if you connect the PC to the HDTV with a VGA or DVI cable, then the PC will either scale the game to fit the HDTV's resolution or the HDTV will scale the resolution used by the PC to it's preferred mode. Good for vector games, okay for high res (VGA) games, but not so good for medium and standard res (EGA & CGA) games.
Okay, to summarize:
1. HDTVs will scale and probably deinterlace to suit their preferred display mode.
2. The less scaling and other modifications which get done to the signal the better.
Therefore connect the HDTV via VGA or DVI and have the PC use the HDTV's preferred resolution.