If you've already gone to the trouble of removing the component to test it, and you've got a replacement available, replace it.
If the measured value is outside +/- 10%, replace it.  Might as well.  If it's an especially expensive cap or something, give it +/- 20% unless you have reason to believe it at fault.
Definitely test ESR.  Excessive ESR is a failure even if the capacitance is spot on.  Of course, you'll have to figure out what it was supposed to be in the first place, and that can be a little difficult sometimes.  For bulk electrolytics, I'd say anything >10ohms is probably "dead" unless you have reason to believe it started higher than that.