Hacker News Reader

Comment by ilaksh | Hacker News Reader

I assume it actually does not understand video, but reads the transcript?

Bard can read images and, being made by the same company as YouTube, probably has access to high quality video embeddings they use for YouTube search, maybe the most sophisticated video search engine on the planet. I don’t know from the article but it could definitely be using the video content directly without a text representation.

For an open source project that actually “sees” videos you can check out https://github.com/PKU-YuanGroup/Video-LLaVA

> maybe the most sophisticated video search engine on the planet

And yet youtube search is such garbage most normal users still use google search to search for videos.

> most normal users still use google search to search for videos.

Citation? This seems unlikely to me. Maybe a lot of users end up watching a video given to them by a Google search. But if someone is specifically looking for a video, I think most probably go to YouTube and use YouTube search (which is more than fine for “normal users”)

> And yet youtube search is such garbage most normal users still use google search to search for videos.

They are the same thing at their core, biggest difference is that Youtube started limiting the number of results but has better filtering options.

Really? What’s particularly bad about it?

Finding only what it's asked for.

Works fine for me:

https://i.imgur.com/JdxFDfX.jpg

https://i.imgur.com/N7Dpb4U.jpg

it's okay. I found tiktok to be better as they search brings your current context for suggestions, user's comments are smartly turned into search links, and they do a better job of letting the user search by transcript

You could test this by asking about an uncaptioned video where the generated captions for any language would be trash (e.g. jiberrish) and using powerpoint slides to convey stuff.

It seems it's only based on the transcript for now. For example when I asked to comment on visual content it mentioned requiring a captions file:

> I'm sorry, but I'm unable to access this YouTube content. This is possible for a number of reasons, but the most common are: the content isn't a valid YouTube link, potentially unsafe content, or the content does not have a captions file that I can read.

No. It just hallucinated[0] about parallel evaluation while this talk doesn't mention parallel evaluation at all. So it has to be more than just the transcript.

[0]: https://g.co/bard/share/9854794a5dc0

Google search certainly seems to have the ability to understand video beyond a simple caption search, so I don't see why bard wouldn't have that same ability.

Reply on news.ycombinator.com