Welcome!
You're totally right, you won't need a monster rig to run the classics. I'll let others go crazy with the *absolute minimum* specs, but you could probably go to a flea market or a pawn shop right now and get something for $30-$40 that can run them. I had much more than just classics running on a (now ancient) AMD Athlon XP 2800+. (That was back around MAME version .117 I think, YMMV with newer versions of MAME.)
The good news there is that since you're using a CRT TV, you're not going to be running HLSL (CRT monitor simulating effects for LCD screens) which drops the hardware requirements pretty significantly, even with the newer versions of MAME, I'd say...
For running through a TV, yeah it's dependent on the inputs the TV has, but you could probably very easily get an old ATI All-in-Wonder or similar graphics card that would take care of your most common non-HD video-output options (composite, s-video, maybe even component, depending on the model...?) Again, since you're not running HLSL, your choice of graphics card matters much less than pure CPU oomph to MAME.
I will say, however, that you may want to consider using a frontend if you're running to a TV. Personally, I think Windows GUI and GUI-based apps don't look too hot, even at 640x480, especially if run via composite cable. Mala is a good choice, but you could even go as simple as its' predecessor, MameWAH. They're pretty easily configurable, don't take up too much in terms of resources (especially compared to the current "big gun" Hyperspin) and they add a bit of flair to game selection.