Hacker News Reader

Comment by richardw | Hacker News Reader

100% LLM’s are very unlikely to get there. They’re fundamentally not suited to thinking like we do. They work on the abstraction of what we’ve written down, which is a good trick but barely hold it together when things get hard/novel.

However, all the confident “it’s fine” votes assume we never invent a better architecture than LLM’s. Given the level of investment and race between countries, it’s not a reliable bet. It’s much, much harder to guarantee safety than it is to find ways it could go wrong.

View on HN

> They’re fundamentally not suited to thinking like we do

LLMs with CoT are Turing-complete. So, theoretically, they can implement any kind of finitely describable algorithm (barring super-Turing computations).

Brainfuck is Turing complete too. But it's not about the ability to implement something, it's about the ability to practically model it. LLMs are magic because the modeling is excessively easy in relation to their capability to infer later.

"They are fundamentally not suited to thinking like we do" stays wrong nevertheless. They are fundamentally suited to everything not proven to be outside their modelling ability.

Okay so by the same logic can’t we say that we can implement human intelligence on a 90s era single core processor? Its instruction set is Turing complete! Now all that’s left is we just have to figure out how the brain works!

Turing completeness applies to a model of computation, not to a physical instantiation of a machine. The stumbling block of "figure out how the brain works" applies more to the argument like the one I was responding to. How a person can know that a general model of computation can't implement the way people think, if we don't know how people think?

The existing LLM training methods on the other hand give the results that are hard to distinguish from "thinking like people," judging by the end results.

So your argument is that scale is also necessary? I can see that, we don’t expect that a single neuron is human intelligence.

That's not the counterargument one might wish, as LLM deep nets are actually implemented on von Neumann hardware, without true understanding of natural intelligence, just our taking inspiration from neurobiology.

The connectionist models are basically a proposed highest possible abstraction of naturally evolved intelligences so it is in retrospect not surprising that passing some hardware scaling threshold they will start doing things that humans and animals do

It's more that formal Turing equivalence plus the Church-Turing thesis tells us that we're not allowed to assume counterarguments based on magic, there's no magic sauce barrier that prevents AI from running on CPU models. The algorithms exist and most of us thought discovering them would be hard.

The empirical surprise was that human intelligence is maybe not that computationally complex after all. (The entirety of academia was basically caught off guard.) That's one not unreasonable interpretation given recent events.

They are fundamentally suited to everything not proven to be outside their modelling ability.

This doesn't seem to make much sense. Surely us being able to prove that something is outside their modelling ability doesn't affect whether it is or not. If I prove something true tomorrow, whatever I proved was also true today.

Or do we have a proof that everything beyond them has already been proved and there are no more proofs left to find?

That's a category mistake.

"Category mistake" outside of kindergarten usually means "I don't like your definitions, but I can't bother to explain which definitions I use."

What do you mean exactly? "Thinking is not an algorithm."?

Bizarro responses. The "usually means" is what people IN kindergarten (or rather those at that level of maturity) mean by category mistake, it's not what informed intellectually honest people mean by it. It seems clear that my correspondent has no idea what it means and is simply lashing out.

It's a category mistake because thinking like we do requires a certain sort of algorithm or one operationally equivalent to it, whereas things that are Turing-complete include Befunge-98 and C++ templates. While something being Turing-complete implies that the cognitive algorithm could be implemented with it, clearly the categories are incorrect.

Note that, while the set of all LLMs with CoT covers Turing completeness, a specific LLM with CoT is only Turing-complete if it has the right weights, but training does not produce such LLMs. Note also that there are no UTMs in the real world since that requires unbounded resources. This is true all digital computers and well as brains so is not a fatal blow to LLMs, but the resource limitations play out very differently in LLMs and brains.

As for the dead troll comment, not only is it a foolish ad hominem on its face, it's quite false -- I'm an atheist and do not believe in souls.

The original claim is "They’re fundamentally not suited to thinking like we do."

> but training does not produce such LLMs

If we are talking about fundamental limitations it should not be an empirical observation: "does not produce" (which is factually wrong, BTW). It should be a fundamental limitation: "can not produce in principle."

I still don't understand what you are talking about when you say "category mistake." I was talking about computational capabilities of LLMs with CoT that their training can exploit, not about Befunge-98.

Erm investing in risky projects requires expected returns that get delivered.

We will soon find out if the party ends or continues to go on.

Hype might get you capital gains. But cash flows matter.

This is a forever problem now.

If/when/how the market crashes mostly doesn't matter, unless we somehow get reset to the stone age. Look up what the capital cycle is. When openAI goes down, someone with real money and assets will buy up the remains. They'll make contracts with the US military and .gov as the government is already hooked. They'll be able to survive the recovery and then instead of us dying in 5 years we die in 10.

When the .com crash happened .com's didn't go away. Bad business models did.

I agree with this. It's concerning where we might be after several more large breakthroughs. None of the technology we have right now seems likely to get to that level

Reply on news.ycombinator.com