It's been my experience that you want to unplug both the power and data connection. Unplugging just the power likely won't have a problem in Windows but it can cause longer boot times in the bios when it 'thinks' it's detecting a drive it can boot from, but can't get info from it (due to not being powered up.) I definitely had this with an extra hard drive I unplugged power from, even though the bios was not set to boot from it, the bootup drive detection process took much much longer because it was trying to find that drive still.
That could be dependent on the motherboard however.