Well, for one, a separate streaming box would only have access to the video it shows. The central multiplexer sees it all.
But yes, a smart TV programmed to do the same is even more unscrupulous given that it can 'spy' on everything it displays, to presumably include the input of connected devices.