It's funny how pushing stringent privacy and security defaults in one domain degrades the privacy and security experience in another domain. My jaded takeaway from the last 5 or so years is that the internet companies (understandably) don't care about non-internet experiences. I empathize with how annoying that reality is because internet technology certainly works locally if you configure everything correctly.. just not how things work by default.
The actual problem though is fascinating and there is a ton to unpack. For one, the internet was always supposed to be zero-trust. Security-by-NAT was an accident not a feature. And IPv6 enshrines this reality (and it breaks my heart when I see tech companies trying to make IPv6 work like IPv4 in the home). The privacy problem is not really an issue if everything is appropriately firewalled and communicating securely. And as you mention, half the IoT things out there only use a central server to get around what is ultimately a problem NAT introduced: devices don't have public IPs and aren't 1st class internet citizens.
Here's how it's supposed to work:
Your ISP delegates you an IPv6 prefix. As the gateway to your home site, your router advertises the public prefix, as well as a ULA prefix to your home devices. Devices construct both permanent and temporary IP addresses from the public and universal local prefixes (at this point a device that needs public internet access has 4 IP addresses). That's just IPv6 so far but the point is that devices have multiple addresses, ones for public communication and ones for local communication.
Once you have the setup above you can do this:
Your ISP provides you a domain (optionally you purchase your own vanity domain, of course) and their nameservers delegate to your gateway device (or possibly some other one, but conveniently your gateway) as the nameserver for your home site. After a device comes online it dynamically registers its desired hostname with gateway, which will now respond to DNS queries with its address. The gateway should also serve PTR and SRV records for your site unicast DNS-SD style. Your nameserver can serve the public record if the request comes from a public IP and the ULA record if the request comes from that prefix. All public traffic stays public and all local traffic stays local.
Since your devices now are publicly routable, they get certs using ACME. If you want to run local ACME on your ULA network, go for it, but you'll continue to run into the original problem that browsers aren't configured to use your local CA by default and getting users to bootstrap that is essentially impossible. In that vein I do wish there was a way to start a browser window for "local" browsing where it only trust one CA (your local one) and thus isn't mixing public and local security domain concerns.
If devices don't want to communicate on the public internet because that's a privacy or security concern, then they simply don't provision themselves a public-prefixed address or add a public DNS entry, etc.
In short, NAT killed the internet and we're still recovering from it.