IPv4s are absolutely needed if you're an ISP/hosting provider or if you run something else than HTTP where there's no concept of "Host" header and it's assumed that 1 IP/port == one instance of the server.
https://tools.ietf.org/html/draft-ietf-dnsop-svcb-httpssvc-0...