Simple solution is just don't log it into your WiFi.
(Now it may not be designed to do that, of course, but we're both responding to <lurchpop>'s hypothetical.)
This stuff doesn't need a butlerian jihad, it needs a DIY surge.
Of course, one could certainly rationally choose to forgo that convenience in favour of increased privacy. I'm just saying the smart features aren't worthless.
Heck, I can DIY it myself but I don't, because smart TVs have the apps and features that are baseline useful. And because it's integrated, I don't need to make sure to turn on some other device to get access.