There really is no reason why a phone couldn't, for example, have a home management app on it that manages all the IOT devices over bluetooth or other protocols directly rather than needing an internet connection.
I made a conscious decision to build out absolutely everything HA-related that I could using Zigbee and Z-Wave devices. I intentionally avoided anything based on wifi and proprietary apps. It's basically what you're describing. I switched from SmartThings to Home Assistant about a year ago, and everything flipped over without any real drama.
As others have said, Matter is another step in this direction with the end goal of making setup a little bit easier; it's still incredibly immature at the moment, though. But Z-Wave and Zigbee are both here now and work fine.
Be that as it may, the issue is also that you can't take your phone and hook up directly to these devices which is the bigger issue for wider adoption, IMO. It's fine for someone like me that has a home server laying around where I can plop in HA. But what about the average consumer who's only device is probably their phone and maybe a smart speaker?
That's more the problem I was talking about that needs addressing.
(And that's kind of the entire point.)
It can operate over WiFi/Ethernet, as well as thread and Bluetooth.
Edit: the implementation is also open source, and you can roll your own
>> This exists, it’s “Matter”
Sadly, I am not sure Matter will solve the problem. Not because it cannot solve it, but because what I see in the industry manufacturers will not solve it.
I have tried several (e.g., Aqara, Google, GE) Matter products that supposed to "just work", but they did not. Every one of the devices I tried failed or made it extremely onerous to function with non-native hubs.
At 3.5% profit margin for the hardware, there is little incentive to truly interoperate. The money is in subscriptions, locking the user into an ecosystem that makes them dependent for the life of the product.
(Caveat emptor - I am an old crotchety, jaded grouch.)
I've got a few cheap Matter light bulbs that I've picked up mostly just to play with, starting a few months ago.
This fleet has several random and forgettable names on the packaging and exactly two (also unmemorable) manufacturers so far.
Pairing is a little weird: It seems to broadly involve a pocket supercomputer with Alexa or Google Home or Home Assistant or whatever, and scanning a QR code.
This QR apparently begins Bluetooth handshake between the light bulb and the pocket computer, wherein things like WiFi information seem to be exchanged.
After that, Matter devices (in my application at least) just live on WiFi.
This all happens without needing weird(er) apps, overseas clown accounts, or manufacturer-specific hardware. It is local. (Well, Home Assistant is local. The others...are whatever hybrids they are.)
And multiple local control systems (like the three I've already mentioned) can each monitor and control each Matter devices directly. There's probably a limit, but it's nice to have these things non-interactively interacting. ;)
And they seem to be working fine. Boring, even. Right now I just have all of my IoT stuff on the same VLAN/SSID as everything else because it is easy, but I have 100% confidence that these Matter devices would continue to boringly Just Work if I were to isolate them to their own VLAN with zero WAN access.
(Maybe that's something I will work on when setting everything up again after the next move.)
Yay?
The problem is that end users suuuuuuuck.
Oh, by the way iOS and Android do everything in the universe to make using Bluetooth absolutely miserable. Which Android or iOS version are you running? Which buggy Bluetooth stack did Samsung saddle you with? Oh, we retired that version of the app 18 months ago, please, for the love of God, update it. And, oh, you're using a shitty cracked Chinese version of the app <facepalm>.
If I make the users connect to the "cloud", I can control the device, the backend talking to the device, and the front end talking to the user. I now know exactly what the versions are, and the Chinese can't pirate the app. The customer support is orders of magnitude easier.
From the perspective of the device developer, the "cloud" is simply a no-brainer on every single front.
You, as a local-only user, simply won't pay anywhere near enough money to make supporting you worthwhile.