It sounds like you're probably a Comcast user? AT&T does this on their U-Verse service, too, but not nearly as badly.
Comcast has a REALLY complicated prioritization/QoS scheme designed to make their network look faster than it is (it's oversold WAY more than most carrier metrics would advise even for consumer access). They don't publish details, but it's something like this:
Highest priority- DNS lookups to their own DNS servers (which IIRC return an ad-serving domain instead of NXDOMAIN)
- speedtest.net (extra "powerboost")
- Comcast's own IP VOD service (when used on your PC, tablet, etc. - when on their STB it doesn't hit the DOCSIS network at all, AFAIK)
- Hulu (Note: Comcast has an ownership interest in this)
- Misc. HTTP traffic not otherwise categorized
- Other not otherwise categorized TCP traffic and some known UDP and bare IP traffic ("general traffic")
- Bittorrent leeching (Seeding is essentially blocked - see below)
- UDP traffic including DNS not directed to their servers. Some well-known VPNs are excepted.
- Youtube, Ustream, Netflix, etc.
Lowest priorityNote: When you transition from leeching to seeding Bittorrent on Comcast, they will throw false TCP resets at you essentially destroying all connections. You can fix this by just blocking TCP resets to your bittorrent ports with a packet filter. You'll end up with a bunch of half-open connections, but it pretty much works.You can get around this in a couple ways, both of which cost money:
- Buy their "Business Class" service: it has a much flatter QoS structure. You also get some other perks like a minimal SLA, better restoration times, no caps (irrelevant right now), allowed to host servers, etc.
- Tunnel everything over a VPN (e.g. IPSec) to another server in a colo somewhere. This adds latency, but Comcast just sees it all as VPN traffic, which they tend to leave alone. This is also a pain to set up if you don't know what you're doing.
AT&T's priority structure is a bit flatter, but it still puts Youtube at the bottom. Since their own IPTV service competes for the same bandwidth (everything runs over the same middle-mile network), it's ALWAYS at the top. Fortunately, it seems to be their transit/peering links that are vastly oversubscribed, not their middle-mile network (apparently 10GbE rings with about 5-10 VDSL neighborhood POPs on them typically). Their network seems to, for now at least, put IPv6-in-IPv4 tunneled traffic squarely in the middle, ignoring the IPv6 content itself, so you can just get a tunnel to e.g. HE.net (free) and repair some broken-ness if you're willing to put up with the additional latency.
You can also use a different service provider if one's available, but I know AT&T pulls this BS, too. "Real" service (e.g. bonded T1, T3, Packet-over-SONET, Metro-E, etc.) will never do this to you, but you probably don't want to see the price on those.