Connect the batteries in series--positive to negative, into a chain. Use whatever wire you need (leads are pretty much the same thing as wire), or just touch the terminals together. It helps if you have a battery holder.
I say to tap (quickly tough together) the connections, because if it's the wrong voltage or polarity, sustained power can damage the bulb. As a general rule, having less voltage will not harm anything, but having too much can burn it out.
Polarity is the direction of the current. If it doesn't light one direction, switch the red and green (plus and minus) leads going to the light.
Dimly is dim lighting; not having the bulb light up completely, but just a little bit. This means it needs morre voltage.
When you're ready to connect it to the power supply, the red wires are +5volts, and the yellow are +12volts. Black is the negative or ground.
Ask more questions if needed.