I've got an 8 year old boy and a 4 year old girl. When my son was 2 or 3, he showed interest in my PS1, so i'd let him try. It was Tony Hawk and at the time, i don't think his eyes could even register the 3dness, ie, his steering was way off, and forget timing jumps as he couldn't tell they were coming.
When he was 4 or so, I got my wife a 2600, which he still plays to this day. His favorite games are pitfall and ms pacman. I think around this time I showed him MAME, and although he still didn't have a ton of coordination, he would sit at my computer for hours. He loves the metal slug series and we rented the ps2 re-release not too long ago.
He's also a N64 nut and regularly plays pilot wings and super mario 64. Recently, he's taken up VF4 evolution on my PS2 and isn't bad considering his age.
I find it fascinating, and rather refreshing, that all he cares about is gameplay. From Pitfall on the Atari, to a modern day fighter, he'll play them all, as long as they have great gameplay.
It's also been cool watching him get better over the years. I reflect on how he played Tony Hawk on the PS1, and now I see him playing THUG 2 and I am amazed at how skilled he's become.
My daughter has shown some interest and she likes to watch me play Frequency. She'll sometimes want to try the 2600, but isn't taking to it like my son did... I wonder if it's just a hard wired thing with boys.
I'm going to build my cabinet after the holidays so I'm sure I'll be fighting them both for playing time once it's done.