This is a long post, but I hope it is informative. I'm attempting to explain a fair bit of circuit theory using language that is fairly plain and easy to understand. About all you need to know is the concept of current and voltage and a tiny bit of freshman algebra. Please forgive me if I seem to gloss over specifics.
LEDs are diodes that happen to emit light. They don't look like conventional light bulbs.
Conventional light bulbs draw current roughly proportional to the voltage you apply across them, like a resistor would (in fact, a light bulb is pretty much a resistor that is allowed to get hot enough to emit light). This means they're fairly forgiving of minor variances in applied voltage. They also can be made to take just about any voltage desired, and 5, 6.3, 12, and 14.4V are all common. If you're off by a little, the current is also off a little from the design value. This means it will either be dim or bright with corresponding small, but measurable differences in longevity.
LEDs conduct next to no current until a certain voltage is applied across them, then they will conduct just about as much as you can shove through it with negligible increase in forward voltage. This is how diodes in general work, but LEDs happen to emit light when they are on (forward biased), which is cool. The downside is that if you apply just a bit too little voltage, they are rather dim, and if you apply just a hair too much, they conduct lots of current, and you let the magic smoke out. The relationship is actually exponential, rather than linear like a resistive light bulb. People refer to the point at which it changes from increasing volage/little current to constant voltage/lots of current as the "knee" due to the sharp curve if you plot a graph.
Typical values for the voltage at the "knee" for a red or green resistor are between 1-2V, usually about 1.4-1.7. For blue and white LEDs, the voltage is higher, sometimes as high as 6V.
Now, you probably only have a 5V and 12V power supply handy, so what to do? You can go to the next higher rail above your rated forward voltage (Vf) and use a resistor to drop the exta. If you go more than just a little above, you get the added bonus that the linear relationship between voltage and current of the resistor swamps the exponential behavior of the LED, allowing for more tolerance on your supply. The only cost is the resistor and maybe 1/8-1/4W of extra heat - still less power overall than a comparable light builb might draw.
So, how do you calculate what resistor to use? Well, up need some information about your LED from its datasheet and you need to know how a resistor behaves. The first is easy: grab the datasheet. The second is also easy: V=IR - voltage equals current times resistance.
We'll wire up our LED with our resistor like this:
O---|>---/\/\/\---O
+ -
Where the + is your power supply + and - is your power supply ground. You can swap the resistor and the LED if you want to. There's valid arguments both ways about where to put it, but either is electrically correct.
Suppose there's 5V between + and -, and our LED is rated as follows (reasonable values below, but check your datasheet):
Typical forward voltage @ rated If: Vf = 1.7V
Rated forward current: If = 10mA
Since we need to go across both the LED and the resistor to go from + to -, we can write:
Vpp = Vf + Vr
5V = 1.7V + Vr
So, we can do a little algebra to find that Vr should be 3.3V. So, now we just need to make that happen. Recall Ohm's law: V = IR. We know V (it's Vr, or 3.3V). We also know that we want 10mA (the rated If) to flow through the LED. Notice that the current has no where else to go but through the resistor if it is to flow through the diode, so we know that I in V=IR = If = 10mA. So:
3.3V = 10mA * R
R = 3.3V/10mA = 330 ohms
And, nicely for us, 330 ohms is a standard 5% value that you can buy at Radio Shack. If you get a number that you can't buy, round up to the next value you can. That will reduce the current slightly resulting in a dimmer LED, but that's better than going the other way and blowing up your LED.
You also need to consider the power dissipation in the resistor. You can know that, though, because Pr = VrIr. So, Vr = 3.3V, and Ir = 10mA, so Pr = 33mW. So a 1/4W or even 1/8W resistor would be fine here. Common practice is to oversize the resistor in terms of power handling by a factor of 2 or more if possible, otherwise they get really hot.
All that said, many LEDs people on here will buy will already have a resistor integrated with them. These will usually be specified as "12V" LEDs or whatever. This is common with the wedge base replacement types, but not discrete LEDs.
Oh, and one last thing about hooking up LEDs backwards. If you exceed the maximum reverse voltage rating (Usually given in the datasheet as Vrm or something like that), you can damage it. Without going into details, the diode will fail to act like a traditional diode and will suffer what is known as "avalanche breakdown" basically conducting a bunch of current until it breaks. It also won't emit light. If you keep the applied reverse voltage under the maximum rated, it won't light up, won't conduct any current (and therefore will draw no power), and won't be damaged.