This simple design when planning and rolling it out would have meant incrementally updating the networking stack to also support v6. Now it turns out v4 and v6 are completely different, and no one has a big enough reason to make the change until everyone else makes the change. Hard chicken-egg problem.
You can try to do hacks like NAT (like you probably do in your home IPv4 network, which breaks/stops any peer-2-peer protocol). The IPv6 version is called DNS64/NAT64, and it breaks even more things, e.g. DNSSEC. Because it not only requires network address translation (lying about your IP), but also rewriting DNS records (lying about signed DNS records).
The only sane way forward is get rid of IPv4 as fast as possible (even if 'fast' means a decade or two).
That’s demonstrably untrue. Sure p2p protocols and NAT devices need to account for NAT, but to imply that they’re impossible to use with NAT is just silly. Many p2p protocols work via NAT all the time...
Or could step back and see that this a terminally ill protocol on life support.
To be fair, I don't think the authors of v6 at the time realized how much friction an alternative IP stack would cause.
(Because it makes things even more complicated, now you'll have not two incompatible but about three things.)
And of course the problem is/was of economics. No incentive to change, v4 is good enough for now, for the incumbents. And even if large companies with new users were running out of v4, they can afford trading netblocks and/or carrier grade NAT devices, both are cheaper than telling others to upgrade (and you'd lose customers/subscribers otherwise).
The sin was committed when IPv4 was made and not initially designed to allow for variable / expanded address space -- it is not IPv6's fault.
Adding an IP Option to IPv4 packets that could carry extra address bits was not an option either -- IP options aren't preserved much at all on the Internet. Furthermore, even if most routers didn't drop IP options, adding "v6" address space via IP option in a packet that old/v4-only devices would nevertheless attempt to parse would have been hell operationally.
granted, IPv6 has lots of complexity/flaws/idiosyncrasies/weirdnesses (multicast, mobility, slaac, ndp, prettyprinting / the colons, extension headers, etc.) that mostly only look good through the rose-tinted glasses of the 90s and significantly slowed down deployment -- and in the end mostly ended up as "difference for difference's sake", but that the transition is difficult is also IPv4's fault for not having a robust address space expansion mechanism.
There is no mapping back from IPv6 to IPv4 (so IPv6 is backwards compatible, but not forwards compatible), but it can't be because the whole point is to have a larger address space.
Hence IPv6 is definitely not backward compatible.
Then there are switches and routers that make assumptions about incoming packets, so we can't do strange bit hacks with IPv4 packets. http://seclists.org/nanog/2016/Dec/29
Switching to a new protocol is the only real choice at this point.