You should CHECK THE VOLTAGE AND AMPERAGE of your lights.
If the voltage is 5V-6V or 12V-14V you can connect them to your PC's power supply, assuming it puts out sufficent aperage, using an extra disk drive connector.
If the voltage is different, you can buy an adapter from radio shack or other electronic stores that meets the correct voltage, and provides sufficent amperage.
You can similarly look at adapters from old electronic devices you no longer use such as toys, keyboards, answering machines, etc and see if one of those is the correct voltage and amperage.
Note: for lights, it does not matter if the voltage is AC or DC