Hacker News Reader

Comment by zten | Hacker News Reader

I realized if someone were to assign me the ticket for fixing this behavior, I would have no idea where to begin with solving it even with this blog post explaining the problem, so I'm very curious to know what the most practical solution is. (They obviously aren't adding "If someone asks you about a seahorse emoji, there isn't one available yet, no matter how strongly you believe one exists." to the system prompt.)

View on HN

Petition the Unicode consortium to include a seahorse emoji.

It takes a strong man to deny what's right in front of him, and if the truth is undeniable... you create your own

If a LLM is wrong about facts, the facts have to change.

I bet they probably are adding that to the system prompt at least in the short term while people are paying attention before looking for a longer term answer.

The system prompts I've seen are absolutely massive.

I find interesting that their blog post on prompt/context engineering kind of stands against their ultra long system prompt. Maybe it is not too specific as in their visual example (too specific - just right - too vague). https://www.anthropic.com/engineering/effective-context-engi... and the system prompt https://docs.claude.com/en/release-notes/system-prompts#sept...

> This attention scarcity stems from architectural constraints of LLMs. LLMs are based on the transformer architecture, which enables every token to attend to every other token across the entire context. This results in n² pairwise relationships for n tokens.

The n² time complexity smells like it could be reduced by algorithm engineering. Maybe doing a preprocessing pass to filter out attending to tokens (not sure what the right term of art is here) that do not contribute significantly to the meaning of the input. Basically some sort of context compression mechanism.

> the most practical solution

Maybe there is none, and this is just one example of a fundamental LLM limitation.

People really really want LLMs to output a highly reliable finished product, and I suspect we're probably never gonna get there. Lots of progress over the past couple years, but not on that.

I think it's much more interesting to focus on use cases which don't require that, where gen AI is an intermediate step, a creator of input (whether for humans or for other programs).

Couldn't you just add a large number of repetitions of "There is no seahorse emoji." to the training set?

Edit: Come to think of it, training on a Q&A format is probably better - "Is there a seahorse emoji? No, there isn't."

Something about how you have to keep repeating "There is no seahorse emoji" or something similar reminded me of the Local 58 horror web series where it seems like the program is trying to get you to repeat "There are no faces" while showing the viewer faces: https://www.youtube.com/watch?v=NZ-vBhGk9F4&t=221

If you had to do this for every falsity in the LLM, there wouldn’t be an end to it.

I agree, but I still suspect OpenAI and other LLM companies do stuff like that, when an example of a hallucination becomes popular.

If I see some example of an LLM saying dumb stuff here, I know it's going to be fixed quickly. If I encounter an example myself and refuse to share it, it may be fixed with a model upgrade in a few years. Or it may still exist.

I think you’re putting the cart before the horse. The article is a good starting point for a root cause analysis but is ultimately speculation.

Before coming up with the solution, I think you’d need to understand the problem much more deeply.

"This behavior is a function of the core AI technology we use, we are unable to resolve this issue with a standard software patch or update at this time.

For the time being this issue can be mitigated by not asking about seahorse emoji.

We are closing this support ticket as the issue is an inherent limitation of the underlying technology and not a bug in our specific implementation."

Reply on news.ycombinator.com