I don't think what you're wanting to do will be very difficult.
I have a JVC television with multiple inputs, but I have a wife who is not good with remote controls. To make everything work, I found an s-video (with audio) switch-box I got at somewhere like Fred Meyer, I think (could've been a video store or a Software Etc), for like $20. It was made to switch between multiple consoles.
It had one set of outputs - Svideo, and L&R audio (RCA type), as well as 4 matching sets of inputs. It had four buttons on the front which you could label. I used 3: DVD, VCR, PS2.
If I understand your question, it would work well for what you need. Just plug your XBox S-video & audio to one input, and plug your computer S-video and audio to another ouput, and perhaps any other devices. Then hook your TV to the switchbox output, and you're set: one-button switching between your console, computer, etc. Also, all sound will go to the TV this way.
What you'll need to hook up your computer audio to the TV is a cable (I think they go for the $5-10 range) that has a 1/8" jack on one side and dual RCA plugs on the other. This would connect the PC sound out to the switchbox. Be careful with this, though; if you turn up the sound on your computer and the sound on your TV is up too far, you can blow out your TV speakers.
Of course, you could also plug the switchbox audio out to something besides the TV, if you so desired. Also, this assumes that you'll have access to the switchbox; mine's about the size of a dictionary lying down.
Purists would probably point to some signal degradation, going through all these wires and a switch, but I never noticed anything.
bm