This is what amazon x-ray does, but it would be nice if you could connect a phone app with the streaming app so you could look up xray data on the second screen.
I love the Amazon X-Ray feature. I wonder if they patented it, because I’m surprised other players like Netflix or cable providers are not offering something similar. I personally find it adds a lot to my viewing experience without being too intrusive.
I wonder how they get the "actors in this scene" data. I suspect mechanical turk but maybe they have the movie scripts.
Amazon owns IMDb. They have been gathering data about actors for literally decades. They probably have well-trained machine learning models to identify actors on screen.
It's really not that hard since they can run it against a narrowed down dataset with the names of the cast.
You can actually cast Amazon video to a chromecast from a smart phone, and then use the smart phone for X Ray. Works pretty seamlessly.