Yes, you can connect the PC Power supply to them. Note that the 4-wire disk drive connectors out supply +5V on one set of wires, and +12V on the other pair. So, if the bulbs are 5 or 6V, use the 5V pair. If they are 12 to 14 volts, use the 12 volt pair. If they are any other voltage, I would consider a different power method.
It is also a good idea to make sure the power supply outputs enough amperage to power them. Most do, but it is a good idea to check.