Joystick and Button controls not working properly

[regulardaddy]

So I bought an old WWF Wrestlefest machine from an auction and the picture went out before I got it home.  I fixed that and finally started it up with working video, only to find that the controls are screwy. 

The start buttons work for P1 and P2, but not for P3 and P4.  P1 can go right, up, down, and punch (button one).  P2 can only go right and down, but can punch and kick (button one and two).  I unplugged the molex connectors and re-connected them.  The controls still didn't work properly, but they changed.  P1 could only go left and down and punch, while P2 could go up, right, down, kick. 

So I figure the connections must be bad.  What's the best way of fixing this?  Should I re-wire it using the existing wires and molex connectors?  Should I replace just the connectors?  Should I replace everything (wires, connectors, buttons, joysticks)?  It's a pretty old game, I'm not sure how good the parts are.