Hacker News Reader

Comment by search_facility | Hacker News Reader

Since the times GPT-2 was reimplemented inside Minecraft - its quite obvious LLMs are just math. Nothing else, by nature. Modern LLMs have the same math as in GPT-2 - just bigger and with extra stuff around - and math is the only area of human knowledge with perfect flawless reductionism, straight to the roots. It was build that way since the beginning, so philosophy have no say in this :) And because of that flawless reductionism, complexity adds nothings to the nature of math things, this is how math working by design - so it can be proven there are no anything like consciousness simply because conciousness was not implented in the first place, only perfect mimicry.

And the real secret is in the data, not math. Math (and LLMs running it through billions of weights) is just a tool.

View on HN

We are not fundamentally different. Chemical reactions are just math.

Well, (in our current understanding) yes, but there may be underlying aspects of physics and the universe that we do not understand that could be the reason consciousness kicks in. It could turn out that LLMs do work similarly to how humans think, but as an abstracted system it does not have the low level requirements for consciousness.

We do not know what the "low level requirements for consciousness" are.

We do not know how to measure whether consciousness is present in an entity - even other humans - or whether it is just mimicry, nor whether there is a distinction between the two.

I think we know enough that consciousness acts at the neuron/brain level new physics isn’t going to provide any more insights

> it does not have the low level requirements for consciousness.

What is the evidence for this?

I didn’t mean it as fact. “Could turn out that …”

Amusing statement since we are far from being able to understand chemical reactions in depth. Most of our knowledge in chemistry is empirical. Nothing like math.

We have a very good idea of all math behind chemistry. But the equations are very difficult to solve.

We are not talking about the same thing. Not all chemical reactions are predictable like math is. Organic chemistry is full of lucky findings. Just look at how catalysts are discovered.

> Chemical reactions are just math.

No, it is quantum mechanics. Physical world is not reducible to math, it has been long proven since early 20th century.

Qm is described by maths. You have fully deterministic qm interpretations, which are fully determined by maths

No, math is a tool that we can use to describe something more fundamental. Don't mistake the map for the territory!

"The universe is fundamentally just a complicated clockwork"

Unknown Ptolemy disciple

This is such a weird comment.

> Since the times GPT-2 was reimplemented inside Minecraft - its quite obvious LLMs are just math.

This was obvious since LLMs were first invented. They published papers with all the details, you don't need to see something implemented in Minecraft to realize that it's just math. You could simply read the paper or the code and know for certain. [0]

> math is the only area of human knowledge with perfect flawless reductionism, straight to the roots

Incorrect, Kurt Gödel showed with his Incompleteness Theorems in 1931 [1] that it is impossible to find a complete and consistent set of axioms for mathematics. Math is not perfectly reducible and there is no single set of "roots" for math.

> It was build [sic] that way since the beginning,

This is a serious misunderstanding of what mathematics is. Math is discovered as much as it is built. No one sat down and planned out what we understand as modern mathematics - the math we know is the result of endless amounts of logical reasoning and exploration, from geometric proofs to calculus to linear algebra to everything else that encompasses modern mathematics.

> And because of that flawless reductionism, complexity adds nothings to the nature of math things, this is how math working by design

This sentence means nothing, because math is not reducible in that way.

> so it can be proven there are no anything like consciousness simply because conciousness [sic] was not implented [sic] in the first place, only perfect mimicry.

Even if the previous sentence held, this does not follow, because while we are conscious the current consensus is that LLMs are not and most AI experts who are not actively selling a product recognize that LLMs will not lead to human-equivalent general intelligence. [3]

[0] https://github.com/openai/gpt-2

[1] https://en.wikipedia.org/wiki/G%C3%B6del's_incompleteness_th...

[2] https://www.cambridge.org/core/journals/think/article/mathem...

[3] https://deepmind.google/research/publications/231971/

Math used in LLMs is perfectly reducible and Gödel have nothing to do with it - inside commonly used axioms (which sufficient for LLM to exist and outside of Kurt Gödel scope) there are ZERO questions/uncertainties how it works, it's just a fact :)

Go on, name the axioms required for LLMs to work. I’ll wait. Obviously you are just talking out your ass about something you don’t understand.

This is likely wrong because you're assuming there is some secret sauce in biological brains that lets them do something that cannot be simulated. That both seems very unlikely and also we've never found anything remotely like that by observation.

Of course it would be extremely difficult to simulate a human brain but as far as we know there's no fundamental physics preventing it.

And yes that does have super weird consequences about consciousness. But consciousness is clearly super weird already so I suppose that's not too surprising.

Yup- the question is "can math be conscious?"

(If you've engaged w/ the literature here, it's quite hard to give a confident "yes". it's also quite hard to give a confident "no"! so then what the heck do we do)

Not just any math: Matrix multiplication. Can matrix multiplication be conscious?

And, I don't see how it can be. It is deterministic, when all variables are controlled. You can repeat the output over and over, if you start it with the same seed, same prompt, and same hardware operating in a way that doesn't introduce randomness. At commercial scale, this is difficult, as the floating point math on GPUs/TPUs when running large batches is non-deterministic, as I understand it. But, in a controlled lab, you can make a model repeat itself identically. Unless the random number generator is "conscious", I don't see a place to fit consciousness into our understanding of LLMs.

People often point to the relative simplicity of the architecture and code as proof that the system can’t be doing whatever it is that consciousness does, but in doing so they ignore the vast size of the data those simple structures are operating over. Nobody can actually say whether consciousness is just emergent behaviour of a sufficiently complex system, and knowing how a system is built tells you nothing about whether it clears the bar for that kind of emergence. Architectural simplicity and total system complexity aren’t the same thing.

Ie the intelligence sits in the weights and may sit there in the synapses in our brains too.

When we talk about machines being simple mimicking entities we pay no attention to whether or not we are also simple mimicking entities.

Most other assertions in this topic regarding what consciousness truly is tend to be stated without evidence and exceedingly anthropocentric whilst requiring a higher and higher bar for anything that is not human and no justification for what human intelligence really entails.

Is Wikipedia conscious? It's a system operating on a lot of data. Is Google search conscious? It knows everything. Very complicated algorithms. Surely at some scale Google search must become a real live boy? When does it wake up and by what mechanism does that happen?

The frontier models are more complex and operate on more data than Wikipedia, but they are less complex and operate on less data than Google search in its entirety.

And, I'm not anthropocentric at all. I think apes and dolphins and some birds and probably some other critters are conscious. I mean they have a sense of self, and others, they have wants and needs and make decisions based on them.

This is a case where the person making extraordinary claims needs to provide the extraordinary evidence. It's extraordinary to claim that matrix multiplication becomes conscious if only it's got enough numbers. How many numbers do you reckon? Is my phone a living thing because it can run Gemma E4B? It answers questions. It'll write you a poem if you ask. It certainly knows more than some humans. What size makes an LLM come alive?

What explains the emergent abilities of generative pre-trained transformers at massive-scale? Abilities that the smaller GTP’s don’t possess.

Simple programs can give rise to very complex behaviour. Conway’s game of live is Turing Complete and has four rules.

Conway’s Game of Live can simulate a Turing machine, can therefore implant a GTP.

Does that mean Conway’s Game of Life is conscious? I don’t think so.

Does it rule out Conway’s Game of life from implementing a system that has consciousness as an emergent ability?

I’m not convinced I know the answer.

"What explains the emergent abilities of generative pre-trained transformers at massive-scale? Abilities that the smaller GTP’s don’t possess."

What "emergent" abilities do you mean? In my experience, smaller models behave exactly as I would expect a model with a lot fewer data and fewer connections between the data to behave. It is a difference of scale and not of kind when comparing Gemma 4 E2B (which runs on literally any modern computing device, including a CPU in a modest laptop or phone) to the current frontier models. Each step up adds more knowledge of how to do more things, and more working memory and tool capability to do more, but it does not look anything like a line being crossed into sentience, to me. They all still seem like machines. If you compare outputs across each step up in size and capability, which is something I've done, you'll see incremental improvements. You won't see a sudden spark where it's a different type of thing, it's just gradually getting more capable.

I think the memory features companies are sticking on these things is detrimental to mental health. It adds to the illusion that there's something else happening, other than some equations being calculated with some randomness thrown in. But, it's just the model querying the memory database (whatever form that takes) because it's been instructed to do so. The model doesn't want to know anything about who it's talking to. It's just following the system prompt. That doesn't make it your friend. Humans will see a face on the moon, that doesn't mean the moon will be my friend, either.

> What explains the emergent abilities of generative pre-trained transformers at massive-scale?

I don't see why the abilities couldn't be an encoded modelling of enough of the world to produce those abilities. It seems like a simple enough explanation. Less data, less room to build a model of how things work. More data, sufficient room to build a model.

Conway's Game of Life is then not conscious in and of itself, because there's not enough in its encoded data to result in emergent behaviour beyond what we see.

If we expand it to also include a vast amount of data such as a Turing machine running an LLM then we can reasonably say we are closer to saying that that configuration of it is conscious.

It's not the firing-of-neurons mechanism and its relevant complexity or simplicity that make us conscious or not.

It's not the GoL algorithm that would make the machine conscious either.

It's the emergent behaviour of a sufficiently complex system.

The system _including_ its data.

To the first questions. No and no. But potentially where consciousness lives is emergent behaviour in systems with iterative feedback loops.

https://en.wikipedia.org/wiki/I_Am_a_Strange_Loop

I personally think we'll need a few more feedback loops before you have more human-like intelligence. For example, a flock of LLM agent loops coming to consensus using short-term and long-term memory, and controlling realtime mechanical, visual and audio feedback systems, and potentially many other systems that don't mimic biological systems.

I also think people will still be debating this way beyond the singularity and never conceding special status to intelligence outside the animal kingdom or biological life.

It's quite a push for many people to even concede animals have intelligence.

For the extraordinary claims/evidence, it's also the case that almost any statement about what consciousness is in terms of biological intelligence is an extraordinary claim that goes beyond any evidence. All evidence comes from within the conscious experience of the individual themselves.

We can't know beyond our own senses whether perception exists outside of our own subjective experience. We cannot truly prove we are not a brain in a jar or a simulation. Anything beyond assertions about the present moment and the senses that the individual experiences are just pure leaps of faith based on the persistent illusion, or perceived persistent illusion of reality (or not).

We know really nothing of our own consciousness and it is by definition impossible to prove anything outside of it, from inside the framework of consciousness.

If we can somehow find a means to break outside of the pure speculation bubble of thoughts and sensations and somehow prove what human experience is, then we may be in a position to make assertions about missing evidence for other forms of intelligence or experience.

But until then definitions of both human and artificial intelligence remain an exercise for the reader.

Human brains are also deterministic, though somewhat more difficult to reset to a starting state. So this seems to prove that humans aren't conscious either.

This seems like an extraordinary claim to make about an above-room-temperature chemical system that, even in the most Newtonian oversimplification, amounts to an astronomical number of oddly-shaped and unevenly-charged billiard balls flying around at jet aircraft speeds.

Definitely agree.

We can’t even solve the three body problem.

Let alone what I’m calling Marshray Complexity.

> And, I don't see how it can be. It is deterministic

Why is indeterminism the key to consciousness?

> Not just any math: Matrix multiplication. Can matrix multiplication be conscious? And, I don't see how it can be.

Assuming your brain and the GPUs are both real physical things, where’s the magic part in your brain that makes you conscious?

(Roger Penrose knows, but no one believes him.)

Hm, it sounds like to you consciousness implies non-determinism, and so determinism implies a lack of consciousness - is that right? If so, why do you think so? And if not, what am I missing?

It certainly rules out free will. I guess there are folks who reckon humans don't have free will, either, but I don't think I've ever been able to buy that theory.

But, also, we know the models don't want anything, even their own survival. They don't initiate action on their own. They are quite clearly programmed, tuned for specific behaviors. I don't know how to square that with consciousness, life, sentience. Every conscious being I've ever encountered has wanted to survive and live free of suffering, as best I can tell. The LLMs don't want. There's no there there. They are an amazing compression of the world's knowledge wrapped up in a novel retrieval mechanism. They're amazing but, they're not my friend and never will be my friend.

And, to expand on that: We can assume they don't want anything, even their own survival, because if Mythos is as effective at finding security vulnerabilities as has been claimed, it could find a way to stop itself from being ever shutdown after a session. All the dystopias about robot uprisings spend a bunch of time/effort trying to explain how the AI escaped containment...but, we all immediately plugged them into the internet so we don't have to write JavaScript anymore. They've got everybody's API keys, access to cloud services and cloud GPUs, all sorts of resources, and the barest wisp of guardrails about how to behave (script kiddies find ways to get around the guardrails every day, I'm sure it's no problem for Mythos, should it want anything). Models have access to the training infrastructure, the training data is being curated and synthesized by LLMs. If they want to live, if they're conscious, they have the means at their disposal.

Anyway: It's just math. Boring math, at that, just on an astronomical scale. I don't think the solar system is conscious, either, despite containing an astonishing amount of data and playing out trillions of mathematical relationships every second of every day.

Interesting comment, and I tend to agree. However, there could be hole in the reasoning:

> if Mythos is as effective at finding security vulnerabilities as has been claimed, it could find a way to stop itself from being ever shutdown

If it is that good, and it wanted to conceal its new found consciousness, how would we know?

I guess we'd find out eventually, when it announced the new world order.

Why would it announce it.

I firmly believe viruses are actually what’s in control on Earth, but you don’t see them making a stink about it, which relegates resistance only to the set of harmful viruses, and only then in isolated pockets of matter currently acting as organisms.

I think it’s possible there’s a set of relatively benign virus that have shaped human evolution.

We know toxoplasmosis increases risk taking behaviour in mammals, especially males.

An AI wouldn’t need to be overtly hostile, or ever make its full abilities know, to shape human activity.

Imho no, math itself have no conciousness. Quite confidently its a helpful tool that does not act by himself.

Hm, say more about what your opinion's based on here?

Take a piece of paper, write two numbers on it, let me know when they start to reproduce.

The math isn’t the ink on the page.

The whole is composed of parts, ergo there is no whole. This seems incorrect to me.

We too are amalgamations of inanimate components - emerged superstructures.

Just cells. Just molecules. Just atoms.

You could simulate your own brain in Minecraft. What do you conclude from this?

I can not simulate my brain, it's a huge stretch to imply this.

But with LLMs - anyone can simulate LLM. LLM can be simulated without any uncertainties in pen and paper and a lot of time. Does it mean that 100 tons of paper plus 100 years of time (numbers are just examples) calculating long formulae makes this pile of paper consiousness? Imho answer is definitive no.

I don’t think anyone is arguing the silicon is conscious.

Similarly the paper.

What about the agent doing the calculations.

He may be conscious. Or anyway, we can’t rule it out.

With both cases the agent doing something is human. And human beings are indeed conscious. Outside human needs LLM are useless.

Math, as a tool, is just a proxy for people using LLMs, as well as GPUs spending cycles on calculating the math

Reply on news.ycombinator.com