The fact that software on my TV knows what I'm watching is mildly annoying; what is more annoying is that it doesn't give me access to that information -- just like my Roku device similarly fails to give me that information.
Ideally, you'd never give your smart TV the wifi password, and connect to a Roku type device that didn't track you. Too bad there is not enough market demand to support the higher price a non-tracking Roku would command.