I've been a homeowner for nearly a decade now and I have to say I sympathize with you, even if for slightly different reasons, mainly the "hassle" factor. I see most of these "smart" things as just one more thing that can break.
Sure it may take a bit longer for me to initially run some wire for speakers (or cost more on a new home build), but once it's set up it is one-and-done. My stereo receivers are a little bulky, but they hide in a corner and the last forever and I don't need to debug their internet connection when I change my Wi-Fi password.
In engineering we talk about building solutions that are as simple as possible, and no simpler. Light switches and door locks are the epitome of this; adding batteries and wireless communication yields a "wow" factor at first, but I cringe at the idea of the numerous ways these can all fail now! I have enough to deal with just keeping up with maintenance on the simple stuff. I'll keep my nightly routine of manually turning off lights and locking doors, thank you.
Not to mention the fact that all of this is changing so fast. Are my HomeKit compatible devices even going to work with whatever phone/speaker/tv setup I have 10 years from now? Am I going to have to pull out all my light switches then? You know what's compatible with the human finger I expect to see at the end of my hand in 10 years? A normal freaking light switch.
I expect that by the time my kids are grown up everything will have settled on a common standard and "smart" devices will be a boring commodity product made by boring brands like GE on my boring Home Depot shelf. At that point I'll probably consider putting it in the wall. Right now I see 50 different one-off Chinese brands racing to the bottom competing with $100 top-of-the-line latest-and-greatest Apple techs and it's all just not worth the effort.