It sucks when something you spent hundreds of hours creating gets stolen and is being used for free while you are trying to make a profit from your hard work.
However, the kind of people who will download (steal) it are not the people who would buy it anyway, so in the end it really doesn't matter.
I used to pirate (steal) all kinds of things. But the games I stole I wouldn't have bought anyway. The music I stole I wouldn't have paid for or I already bought once (and lost or the media was damaged). The movies I stole I had either already seen or weren't available to buy. Is it justifiable? No. Would someone have profited from me if the piracy was stopped? No. In fact chances are if the media is worthwhile, I might tell someone who WOULD buy it that it was worthwhile and it could lead to a sale. Of course, not everyone is like me, some steal simply because it saves them money they WOULD otherwise spend.
However, piracy for the sake of giving stuff away for free is not the same as piracy that someone else profits from. If someone was selling my books for money (or trading them for something of value), I would be way more offended and far more inclined to pursue charges.