Anti-virus, maybe, the ISP definitively needs to step up and just shut off people internet when large amounts of bot traffic is detected. IPv6 is going to do nothing, because right now you're getting scrapped/attacked/DDoS/whatever with a single request from millions of IPs at once.

the root of the root of all evil: NAT

How exactly does IPv6 "stop crawlers".

If anything, it will make it harder to block due to the vastness of the IPv6 address space.