We replaced records with tapes, tapes with more tapes, and more tapes with CDs before they, too, disappeared from the shelves in favor of streaming. Except that some stalwarts have successfully resurrected vinyl.
We replaced AM with FM, and analog radio with digital radio, then streaming. We replaced broadcast analog TV with digital, then cable and satelite, then streaming. Mostly.
None of these changes were backwards compatible, and all of them were meant for the general public. They took a while. They were successful.
The problem with IPv6 is that you don't get benefits. If the designed protocol needs an equivalent of big bang, it's doomed. ASCII->UTF8 didn't need big bang. x86 to Itanium needed big bang.
v6 is about as backwards compatible with v4 as it's possible to be. If you have a way to make it more backwards compatible then I'd love to hear it, but when I ask this all I ever get are things that don't work, or things that v6 already does.
But for another, the v4 space is available as a subset of the v6 space:
$ ping 64:ff9b::8.8.8.8
PING 64:ff9b::8.8.8.8(64:ff9b::808:808) 56 data bytes
64 bytes from 64:ff9b::808:808: icmp_seq=1 ttl=113 time=9.82 ms
That's from a machine on a network with no v4, and it works fine. I can reach v4-only sites from it too. I could even do this using v4 addresses if I wanted, but if I showed you the output from that you'd just claim I was using v4.127.0.0.1 needed to be a valid IPv6 address, along with all the others. Pick a particular prefix, say 0...* and any address with that would be extended to 128 bits. That would have been backwards compatible.
There is no possible way to design an address protocol with bigger addresses than v4 that a) makes v4 forward compatible with it, and b) can actually work. Feel free to suggest one.
> .0.0.1 needed to be a valid IPv6 address, along with all the others. Pick a particular prefix, say 0...* and any address with that would be extended to 128 bits
That prefix is ::ffff:0:0/96. 127.0.0.1 is ::ffff:127.0.0.1 (::ffff:7f00:1). 30 years and you still haven't realized v6 has this?
That doesn't mean that the limits of the old design won't hit anyway and force a switch off it.
The problem is that you can't use those bits to expand v4's address space, without taking all of the same steps v6 needed to do. v4 has no mechanism to get v4 hosts to understand extra address bits, wherever you put them.
Oh, that and the fact that IP addresses are stored in many more places than just the v4 packet header. Consider DNS, DHCP, ARP, gethostbyname(), struct sockaddr_in, databases using VARCHAR(15), etc etc etc. The packet header is only a tiny part of the story.