800x600 will be ok, but not great. Tempest was a vector title, so no pixels per se. However, it was a color vector title, so the crt still had a resolution of its own. With modern, small dot pitch crts and high res graphics cards driving them, you can actually have a sharper looking display than the original, albeit without the "pop" the vector monitor provides.
I think the original dot pitch of the display calculated out to something equal to 1280x1024. It's usually best to aim a little high on the side of resolution, and then use a higher "beam width" setting to get it bright and colorful.
It will still be plenty playable and not look too bad at 800x600, but you would likely see some "jaggy" lines.
RandyT