I do expect when I delete, say, Steam, it won't leave 20 GB of games I can't play sitting around in a totally invisible place. Similarly, when I delete some video or sound editing software, I don't expect it to leave several gigabytes of samples and filters lying around. Both of these are real situations I've encountered when people came to me asking why their hard disk was so ridiculously full.

I can understand not deleting things out of ~/Documents, but a lot of stuff that goes in the Library folders is not what users think of as data that should outlive the application.