Hacker News Reader

Comment by glenstein | Hacker News Reader

I haven't seen Anora so I'll give you that one, but you cited that as if it was just one of many examples, when in fact I think it's the only one, as all the other descriptions seem reasonable.

Originally the problem was supposedly that it would hallucinate complete and utter gibberish, but now here we are quibbling over one example and insisting that maybe it's not quite as good as alternative descriptions.

The gap between what was produced and what you're looking for is small enough that I think it could be covered with some slightly tweaked prompt instructions.

I'm not saying you're wrong but want to note how the goalposts keep seeming to shift whenever we talk about these capabilities.

View on HN

I'm not Alupis. I can't and am not trying to speak on their behalf. I'm therefore not moving the goalposts they established. I'm making my own related point.

That point is that the information provided above about these movies is worthless. It does not add any new value beyond what would already be available in the streaming interface. Several of the descriptions are nothing but the genre and one person involved in the making of the movie. And yet even with these descriptions being incredibly short and vague, they still manage to contain at least one misleading summary.

I'm aware that you're a different commenter but you are addressing yourself to a comment that was in reply to them and therefore not necessarily appropriate to measure such a comment against entirely new criteria that you want to bring into the conversation.

Despite your protestations to the contrary, these descriptions seem perfectly fine in that they're accurate and meaningful. And it if you want to start getting fast and loose with all kinds of new extra criteria and requirements for what it's supposed to do, they all seem squarely within the reach of the capabilities on offer, with some prompt tweaks.

>these descriptions seem perfectly fine in that they're accurate and meaningful

The description of Wicked doesn't mention either The Wizard of Oz or the Broadway musical. So yes, the descriptions don't contain obscene mistakes like calling Wicked a courtroom drama. If that is enough for you to call these "accurate" while ignoring the vagueness or the 1 in 10 failure rate on the Anora description, fine by me. But you must have some weird definition of the word "meaningful" to apply that to descriptions like the one of Wicked. That simply isn't a helpful way to describe that movie.

The comment thread you're at the end of started with this:

> I would expect nothing but hallucinations and nonsense coming out of any LLM regarding recently-released movies (aka. the ones you often find on flights).

The comment that replied to it (the one that you're arguing against) provides evidence that proves it wrong. You are correcting someone who isn't incorrect, and I think the person you're responding to is very justified in saying you're moving the goalposts here.

A reply downthread is not an endorsement of everything said upthread. I'm happy to discuss the points I made, but I’m not going to be made to defend something I didn’t say.

Well if you're not endorsing what was said upthread, then your comment is a complete non-sequitir. The parent comment said "LLMs can't give movie recommendations for recent movies because they'll hallucinate or spout nonsense", the next comment responds with a list of accurate movie recommendations, and then you come in and say this:

> Those descriptions are less detailed than the information you will see on basically any streaming interface and yet it still manages to not being very good.

The points you made were not relevant to the discussion at hand. It's like if people were having a debate about where to find the best tacos in town and you stepped in to say "tacos aren't as good as hamburgers, you know" and then got upset that nobody wanted to debate that point with you. It's not everybody else's fault if you don't understand how conversations work!

I don’t know why you are letting that one reply define the bounds of this conversation. My comment was directly relevant to the first comment in this thread and the comment I was replying to.

> I haven't seen Anora so I'll give you that one

It was literally the first movie in that list.

You tried making a counter example and the first part of it was already wrong.

That’s the point. Not that it _cant_ give good answers, but whether it does or not is a crap shoot.

Now to analyze how correct it was we need to verify each movie it gave… It’d be faster just to read the movie descriptions.

You're still trying to imply that the list as a whole is as inaccurate as that one particular example.

And I think that's quite obviously not the case, most, probably every other example on the list is just fine.

I’m saying that because of the one incorrect example, I can’t just assume that the rest are accurate.

I now need to either trust a machine that I know gives incorrect information (as demonstrated by the first example) or I need to verify each example.

> probably every other example on the list is just fine.

Why don’t you check IMDb and let me know?

While you’re at it, don’t think about how much faster it would’ve been if you just looked up popular recent movies on IMDb or rotten tomatoes.

For comparisons, Wikipedia's opening paragraph for Anora reads:

"Anora is a 2024 American comedy-drama film written, directed, and edited by Sean Baker. It follows the beleaguered marriage between Anora (Mikey Madison), a young sex worker, and Vanya Zakharov (Mark Eydelshteyn), the son of a Russian oligarch. The supporting cast includes Yura Borisov, Karren Karagulian, Vache Tovmasyan, and Aleksei Serebryakov."

I haven't seen the film, but it doesn't seem incompatible with ChatGPT's briefer description.

You see the issue here, right?

Instead of just checking with a first party source, you ask a statistical guessing machine for an answer.

There was a disagreement about the answer, so we needed to dig deeper.

You bring up Wikipedia, a 3rd party source of information. That description could also be wrong (it’s probably not, but stick with me)

Instead of just checking with a first party source (IMDb is very easy to search on), we went through several layers of obfuscation.

This was an issue for Wikipedia early on, but it has citations, at least. AI doesn’t and doesn’t have an army of people constantly fact checking every answer generated either.

There’s no benefit to asking AI for information like this. Especially since the in flight summary has accurate information that’s more than “drama, sex worker, cony island”

Maybe something like perplexity is better, since it has citations, but I haven’t tried it for very long yet.

I _have_ seen Anora and I think that description is perfectly fine. It certainly isn't "hallucinations and nonsense" which is what the parent comment is claiming. What part of that description do you consider "wrong"?

Reply on news.ycombinator.com