I had a similar problem when I was hacking a original psx pad...
I scratched the brown color covering contacts so that I can see the metal contact... and there is only a very small contact for me to solder....
I solder all the wires... it'll stay on... but if I yank it with some strength, it'll come right back off..... (well... when I mount the hacked circuit board, it's hard to avoid some yanking... which means it'll come off.....
so... what I did is... I soldered the best I could... and then carefully test all connections.... by connecting it to the psx and try touch the wires with the ground and see if the button is hit.... (after verifying all soldering points are good.... and unplugged the circuit board from psx... of course...)
I put elmer's glue on top of the circuit board all over the place...
the elmer's glue are those general purpose white glue I buy from home depot.... DO NOT... I repeat... DO NOT plug it to the psx before the glue is dry !!!!!..... cos when the glue is not dried, its wet and will short all your circuits.....
after its dried... the glue will help holding the wires on the circuit board..... (the glue will turn transparent after its dried... I left it alone for 2 days to be sure....)
try this link to see a pic.....
http://www.geocities.com/hyiu/Arcade/picsand click on
psx_circuit_board.jpg
you should see how the glue is holding the wire as well as the connection.... (well... those elmer's glue is not ultra strong, so, I still try not to give those wires any tension.... but so far so good.....)

hope it helps....