The problem is that we are too deep in this hobby to give you a simple answer.

The x-arcade 2 player kit (currently $70) purchased directly from them is probably the best choice for an all in one kit for a noob.
http://shop.xgaming.com/collections/arcade-parts/products/two-player-complete-do-it-yourself-arcade-kitNothing is spectacular about it, but nothing is bad either. If you have any problems, you can contact them for support.
What I really recommend, but is more complicated and more expensive:
For the wiring, get a generic JAMMA harness off ebay and hack the big connector end off of it:
http://www.ebay.com/itm/JAMMA-Wiring-Harness-with-Wire-Id-Label-Arcade-Video-Game-Multicade-/121753778577?hash=item1c59173991:g:4FQAAOSwwE5WXggFFor balltop joysticks, I recommend either the Omni switchable 4/8 way (if you're going to be able to flip up the control panel to switch it) or the cheap Zippy with the more expensive Hand Candy balltop option.
After using standard issue balltops (with seams) and the hand candy which are perfectly smooth, I will never go back.
http://groovygamegear.com/webstore/index.php?main_page=index&cPath=65Alternatively, the J-Stick from Ultimarc is pretty much the same stick as the Omni without the little handle modification. You can keep the bolts on the bottom of it loose and switch from 4 to 8-way also.
http://www.ultimarc.com/store/section.php?xSec=6They are both Sanwa JLW sticks which are good all-around sticks. The more popular Sanwa JLF has a super light spring that I describe as "dainty". It barely has any resistance when you push it which is drastically different from what I grew up with. Of course your son probably has no such expectations.
To the casual player who is going to play Pacman for a few minutes and then switch to a different one, the 4-way games in MAME will play fine with an 8-way stick.
The 4/8 way switching thing is more for people who are really into the original games and want it to "feel right".
It does make a difference in how well the game plays though. On certain games you seem to have better control over the character with a 4-way stick (pacman doesn't miss the turn you meant to take, Mario seems to exit ladders easier, burger time is still a ---smurfette---).
For the interface I recommend an I-pac 2 (which shows up as a keyboard) from ultimarc if you're going the pi route.
I haven't messed with a pi, but have read that there is an issue with MAME for pi only recognizing 16 inputs from the GPWiz.
Perhaps someone who has used one with a Pi can comment.
If you go the PC route and order from GGG, then get the GPWiz which will show up as gamepads.
Personally I prefer keyboard encoders because they are compatible with every oddball old emulator, but some combinations of the MAME default keyboard keys can trigger windows hot keys or desktop rotation.
The first thing I do is change those keys on the encoder and remap MAME, but for you an interface that shows up as a gamepad will mean you don't have to deal with any of that.
For buttons I recommend the cheapest
concave ones from whatever vendor you order from.
Order a button wrench with them.
EDIT: attached pic of stuff from GGG. There is an option to add a wiring to the GPWiz for $19, but I'd still go the ebay JAMMA harness route because I like different colored wires.
green buttons are for coin. you may want white or something else. You may want P1 side red and P2 side blue.

DOUBLE EDIT: Forgot the button wrench in the cart below.....don't forget the button wrench!
