Whenever they tell me my smart lock can be hacked I respond... "yeah, and someone can also just take a brick, smash my window, and go through that way."
My origional comment might be a bit simplistic because In my case in order to not be caught they would have to hack not just my lock but also have to hack my security system network (separate from my home Wifi) and disable the alarms and camera.
But you're right not everyone has layers.
Although one could argue a traditional non-smart lock is not hard to "hack" and leave no evidence either.
Can you see our point now?
Also, most smart door locks are operated over a protocol called ZWave which has a limited range. When people talk about smart lock hacking they are usually talking about RF spoofing of the ZWave signal. You can, of course, hack the bridge device if the bridge device is on Wifi.