Yep, this is an oft-used argument for why Google might not aim to penalise sites for having spammy backlinks pointing to it.
I think that the recent high profile cases have been a tad more obvious since a bit of research can find that it was indeed that company (or their SEO company..) who built the links.
I'd imagine that in general, Google would aim to simply discount spammy backlinks for the reasons you mention.
Especially since tools like xRumer can build 100,000s of backlinks per day, and various other ones (ScrapeBox etc) can build 10,000s per day...