I can't really attest to this particular product, but if I were to make a switch I would just disconnect the ground and 5v line from the usb device not in use. Even for passive usb devices, if that ground wire doesn't get a specific range of power back the device isn't detected. The disconnection of the serial lines aren't nearly as important, because without power they can't really transmit anyway.
I think it should work... usb, for all it's mystery is a 5v powered serial port, but the only way to know for certain is to try it.