Magento, it sounds like the directional wires you are hooking up either have a voltage on them or ground potential on them. A couple things you can try to troubleshoot:
Hookup just the ground and +5 to p360. Verify with a multimeter using the probes against the p360 connections for ground and +5. Then take your up button wire, and just press it to the ground connection on the p360....this should "activate" the button, and you should see downward movement on the screen. If you don't see movement, then while down is pressed to the ground, check the voltage of the +5. If +5 is still there, then your pads aren't hacked correctly....remove p360 completely, and verify each directional connected to ground works. If the +5 drops when you ground the up wire, then you can't use +5 from the pad.
One more note. If you use +5 volts from an external wall adapter, you should NOT connect the apadter's ground and the pads ground to the same p360 connector. Place a diode inline with the hacked pads ground to the p360. This just ensures that you don't get any voltage back from the adapter into the pads ground.