Prescription bottles, bills, product packaging, letters, used condoms, expired drugs, correspondence of all sorts -- including links to organizations like the NRA or ACLU, incriminating materials such as joint roaches or needles, gossip inducing materials such as needles (for legit drugs), sex toys, gay porn for an in-the-closet person, medical records, financial records, etc., etc., etc.,
You can find out just about everything you want to know about most people by going through their garbage, which is why it is done by law enforcement and papparazi all the time. You don't keep it because it would be impractical, and oftentimes gross to hold onto it, but that doesn't mean that the average person has no interest in keeping their garbage private. You are required by law to dispose of garbage -- it can't be kept on your property, and generally the only option is putting it at your curb for pickup. Burning is prohibited (not to mention extremely inconvenient) for almost everybody.
But relinquishing possession doesn't mean you automatically relinquish a legitimate privacy interest. Is it okay for the mailman to go through your mail simply because you dropped it in your mailbox?