Why do you want a switch to power on/off all your LEDs? Why not have them powered on with your computer, in which case an LEDWiz or similar controller works great. The 5V feed driving the LEDWiz comes from your computer's power supply. You turn the computer off (it's power button, power strip, etc...), the LEDs turn off.
If you still want a button as your go between (not sure why), then simply wire the switch between your 5V source and the LEDWiz controller. Assuming your 5V source is powered on, your switch will either complete that circuit or not, depending on its state.
If you purchase a button/switch with it's OWN LED, you have seperate tabs on the back of the switch. Some tabs control the switch itself, and the others the LED state. This LED will need to have a power source as well (and could also be your computer's power supply (5V or 12V depending on the LED and what its expected supply voltage)