> Google and my employer deal with massive amounts of spam emails and gets it right 99.9999999% of the time.

Emails are literal treasure troves of data and are easy to pull signals out of for determining spamminess. HTTP GET requests, on the other hand, aren't - and there's no way to associate them with other spam-connected properties (like you can with click-through domains in emails) besides by blocking entire IP ranges for the ASN not moderating their customer base.