Sure, now you just need a way to validate the public key and IP address genuinely belongs to the claimed identity. Could use a certificate?
You must be a TLS expert because saying we need to check if a public key "belongs to the claimed identity" is creating a problem where none exists. The public key is an identity, that's who I want to connect to. In the Wireguard example, it really is that easy.
With https this key exchange mitm aspect is acknowledged and that is where most of the complexity comes in. Since a client typically connects to a website without prior keys, we need a trusted third party like a CA or trust chain to verify were connecting to the domain we intended to connect to.
Not staying tls isn't bloated, but to be fair, it solves a more complicated problem.
In a lot of cases the IP comes from a configuration file. That's assumed to be secure. In some cases it comes from an authority. If I'm trusting the authority for the IP address, why can't I trust them for the identity as well? I have to trust a different authority for that? That is twice as complicated as necessary.
DNS should really be about which identity owns a name. We are starting to get there with DNSSec, once that's sorted out we could easily return `<public key>@<ip:port>` entries in authenticated DNS records. The root server (transitively) claimed that this public key represents this domain name, and it was last seen accepting connections at this IP and port.
The problems with make name lookups yield certificates are all real-world, pragmatic, people-based issues:
* The deployed base of DNS software, which includes middleboxes of all shapes and sizes that pay attention to DNS, is hostile to things that look like DANE lookups, so DANE lookups (really, all DNSSEC lookups) have a high failure rate, so high that anything using DNSSEC needs to have an alternate path to building a secure channel without DNSSEC.
* The entities that ultimately decide what DNS lookups are going to return (note well: those entities are rarely ever the people who "own" the zones themselves) are themselves firms with roughly the same shape as certificate authorities, but with none of the accountability mechanisms; browser vendors had to force legacy CAs to adopt Certificate Transparency and no such leverage exists for the DNS.
* For bonus fun, add in that ultimately most of the (~all of the popular) TLDs are de jure government controlled --- and governments have a lot of practice exploiting their control over DNS for policy ends.
DNSSEC is a dead letter. Deployment in .COM actually WENT DOWN within the last 24 months, and it was trifling to begin with. The most common experience large tech companies have had with deploying it is "falling off the face of the Internet for several hours due to misconfiguration". Stick a fork in it. It was a reasonable idea that, like many reasonable ideas, has turned out to be completely impractical in the real world.
For the vast majority of use cases, this cannot be done, hence certificates (with CA)