Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by kinduff | Hacker News Reader

Full thread

kinduff·

Do you think the performance can be improved if the representation of the level is different?

I've seen AI struggle with ASCII, but when presented as other data structures, it performs better.

edit:

e.g. JSON with structured coordinates, graph based JSON, or a semantic representation with the coordinates

View on HN

In the limit case, to an actual general intelligence, representation is superfluous, because it can figure out how to convert freely.

To the extent that the current generation of AI isn't general, yeah, papering over some of its weaknesses may allow you to expose other parts of it, both strengths and other weaknesses.

A human can easily struggle at solving a poorly communicated puzzle, especially if paper/pencil or something isn't available to convert to a better format. LLMs can look back at what they wrote, but it seems kind of like a poor format for working out a better representation to me.

I found some papers [n] about this. And I think the answer is yes, the format matters asnd hence the representation.

I wonder if the author would be willing to try with another representation.

[1]: Does Prompt Formatting Have Any Impact on LLM Performance? https://arxiv.org/html/2411.10541v1

[2]: Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding - A Survey https://arxiv.org/html/2402.17944v2

If it struggles with the representation, that makes it an even better test of the AI's thinking potential.

I'm not sure. Adding superficial difficulties to an IQ test for humans doesn't (necessarily) improve it as an IQ test.

These models can “code,” but they can’t code yet. We’ll know that they can actually code once their performance on these tasks becomes invariant to input representation, because they can just whip up a script to convert representations.

Reply on news.ycombinator.com