Terrence Tao: AI Is Teaching Us Something Uncomfortable About Our Own Minds [video] | Hacker News Reader

Terrence Tao: AI Is Teaching Us Something Uncomfortable About Our Own Minds [video]

youtube.com

18 pointsthelastgallon11 comments

Going off on a tangent here: for over a decade, ever since I got an inkling of the power of neural networks, I've argued that privacy is the key to maintaining a broad diffusion of wealth in the modern electronically networked world.

What distinguishes human intelligence from LLM intelligence is largely information that the latter hasn't mopped up. So the information that the former is uniquely privy to is what gives it economic value.

I despise the "next token predictor" framing.

In order to predict the next token in this line "The solution to your problem is..." the entity needs to be able to solve your problem. Which is well beyond what people think of when they think of a token predicting statistical sequence.

The statistical sequence description is ridiculously incomplete.

Next token predictor is as useful as saying that humans are just atoms and neurones.

It is technically true but meaningless.

An LLM agent is literally Turing complete. There is nothing it cannot compute.

Anytime you hear someone saying "it can't do this, or that..." they are factually incorrect.

The denial around these issues is beyond creepy: AI has clearly hit some tender nerve in the human mind.

Source? Turing completeness is a very specific mathematical definition, it would surprise me if a neural network met it. Real-world computers are only Turing complete if you assume that they have unlimited memory space -- which is a reasonable approximation, but technically not true.

Yes.

If we allow a file with unbounded storage (like your exception for physical computers) then an agent can easily perform the necessary state transitions and tape reads/writes.

the entity needs to be able to solve your problem.

People have fixed software bugs with zero knowledge of what the code is doing and no knowledge of the programming language at hand, purely by statistical pattern matching. Solving the problem with no understanding of it beyond the idea that whatever it was, the right answer was probably something that kind of matched the kind of patterns found in things they'd seen before.

So what does it mean, "to be able to solve your problem"?

> People have fixed software bugs with zero knowledge of what the code is doing and no knowledge of the programming language at hand, purely by statistical pattern matching.

Is that even true? Beyond an extremely simple example?

Yes, it is. Beyond simple examples, I've fixed segfaults by identifying the assembly where it happens, tracing that back to the lump of source where it happens, guessed which piece is accessing memory and deducing from surrounding code how it was meant to be protected against.

In that case, the pattern matching was against every segfault I've ever fixed, which is a lot, and traipsing through assembly, which is also a lot. Lots of experience in tracing back to source code. So did that. Matching patterns. Didn't understand the language, don't know what it's trying to do, but I know what memory access is (because I've seen the pattern "access memory" a lot in my life) and this piece of code must be matching that pattern; whatever this code is, it must be touching memory. Now scroll up and down to look for similar code in other places, that I also cannot read but I can see when things look the same; match the pattern of pieces that work with this piece that doesn't. Oh, this looks the same but it has this extra piece; part of the pattern matches, part does not. Matching patterns. What if I copy it, paste into the bad place, apply a single change to match the symbols I see in the bad place, and see what happens... fixed.

It's not something I do every day and it's not nearly as good as actually understanding it, but the problem was solved and I did not know what the code was trying to do or how to read it.

All the above aside, surely it must be possible, because we have machines that do exactly that. They're made of statistics and patterns and they're doing it. Right in front of us. There is nothing else in there and they're doing it right now.

Yeah... I think you just made that up.

Done it myself.

The fact that you have a long history in the business makes me think we're either talking about wildly different things and this is a mismatch of language and context, or you're deliberately trolling me.

It can take a good few years to get to the point that one no longer has to think and understand in a concrete language. There are whole sets of common mistakes that get made across languages and tools. I've fixed errors in Pascal, which I do not read, because the person writing it was clearly thinking in C and made a mistake in their C thinking which they then translated into Pascal. Luckily for me they used common C naming and layout for that piece so I matched the whole thing up to C patterns.

Copy paste errors are easier ones to see; cases where replicated code is clearly meant to be different in a few keys places or values, and whoever did it just missed one. Array errors are common, no matter how the array notation works in language du jour. The list goes on.

All that aside, we know it's possible because there are machines doing it right now. They are made of stats and patterns. There is nothing else in there.

Reply on news.ycombinator.com