Hacker News Reader

Comment by cudgy | Hacker News Reader

The ability to create novel solutions without a priori knowledge.

What would you consider "priori" knowledge? Issac Newton said "If I have seen further, it is by standing on the shoulders of giants.".

I am struggling to think of anything that can be considered a solution and can be created without "priori" knowledge.

I think you're mistaking "a priori" with "prior." A priori is a philosophical term meaning knowledge acquired through deductive reasoning rather than empirical observation.

Thanks for the explanation.. It still does not make sense to me.. A novel solution without deductive reasoning or a novel solution without empirical observation?

To be honest, I don't think their definition of intelligence is very coherent. I was just being pedantic.

But if I had to guess, I believe they'd argue that an LLM is basically all a priori knowledge. It is trained on a massive data set and all it can do once trained is reason from those initial axioms (they aren't really axioms, but whatever). While humans, and actually many other animals to a lesser extent, can make observations, challenge existing assumptions, generalize to solve problems, etc.

That's not exactly my definition of intelligence, but that might be what they were going for.

Humans derive their ideas from impressions (sensory experiences) and the ideas they form are essentially recombinations or refinements of those impressions. In this sense, human creativity can be viewed as a process of combining, transforming, and reinterpreting past experiences (impressions).

So, if we look at it from this perspective, human thinking is not fundamentally different from LLMs in that both rely on existing material to create new ideas.

The main difference is that LLMs process text statistically, while humans interpret text in context, influenced by emotions, experiences, biases, and goals. LLMs' interpretation is probabilistic, not conceptual.

Additionally, revolutionary thinking often requires rejecting past ideas and forming new conceptual frameworks, but LLMs cannot reject prior data, they are bound by it.

At any rate, the question remains, are LLMs capable of revolutionary ideas just like humans?

But the major difference between the human perceptual apparatus and data fed to an LLM is that humans are, in a linear temporal fashion, experiencing a physical world that exists outside of our perception. Our observations aren't just large volumes of unstructured data with purely statistical relevance to each other. Instead, we attempt to model the world via objects existing in relative position to each other and events occurring at various point in a timeline. The result is a complex model of cause and effect, actors and things being acted on, etc.

In that way, my dog is far more intelligent than LLM, in that he has a mental model of his world. An LLM is only intelligent relative to a human actor, and so it is no different than any other technology that humans have created to pursue their own ends.

> The ability to create novel solutions without a priori knowledge.

If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people.

So I disagree. There's a lot of ways to be intelligent. Not just the research and scientific type.

It takes novel solutions to walk down the street, interact with folks, dodge random incoming obstacles, respond to comments, and a bazillion other things almost everyone does all the time.

I probably agree that most people aren't engaged very often and even when they are they suck at being awesome but that really isn't the bar being mentioned here.

> It takes novel solutions to

Right

> but that really isn't the bar being mentioned here.

Yes, the bar was "novel solutions WITHOUT priori knowledge"

So you've changed the definition. Please re-read what I disagree with and it's not just the novel part i.e. if I read it all from the Internet and copied it to be successful then that fails this definition.

LLMs can interact with folks and respond to comments, which are the things on that list you should judge someone without a physical presence on.

> If you go by that then a lot of people (no offense) aren't intelligent. This includes many vastly successful or rich people.

I think most people agree with this statement.

Creating novel solutions have nothing to do with academic. Almost everyone encounters unexpected situations, that they have to think about to solve. It may not been novel as a whole, but for the person, it is.

Reply on news.ycombinator.com