I think the biggest issue that you may run into is destroying the speakers by underpowering them. If you use an underpowered amp (which you may well be doing with using computer speaker amps), you'll probably have to crank up the amps to get some decent volume out of them. Now, when you crank up the amp too much, the amp won't be able to put out the power, which introduces a whole lot of distortion into the signal. Think of a sine wave, then lop off the tops and bottoms of the waveform. (This is known as clipping) This is what you're sending into the speaker. Notice how at the peaks, the direction of wave abruptly changes? Speakers don't like that.