Could be the power supply fan. Stick a Q-tip in the back of the power supply fan grille so that the fan can't move and power it up for a few seconds to see if the whine comes back. Make sure to pull it out after you've made your determination.
If it's not the PSU fan, it could be a bad cap somewhere. Are you getting any display weirdness when the machine is on, or issues with no video when you turn it on sometimes?