What you're talking about exists: it's called DANE: https://en.m.wikipedia.org/wiki/DNS-based_Authentication_of_...
Although I always thought it would be a nice feature for security conscious folks to be able to ennable. Or go ahead and use it on more sensitive sites only, e.g. banks.
https://www.cloudflare.com/en-au/learning/dns/dns-cache-pois...