Hacker News Reader

Comment by BulgarianIdiot | Hacker News Reader

Has GPT ever told you "let me think" and then you notice it actually pause for a few seconds? No. Its output is fixed cost. GPT can't think in depth because it's not allowed to. This is something I already covered above by noting that current NN models are a DAG. We're already experimenting with variations that can iterate internally, and therefore think deeper, using still their existing knowledge. And the results change dramatically.

You're also not right that GPT can't produce anything novel. Here's a tiny, modest example:

https://sumplete.com/about/

I won't waste my time, as I see you didn't read what you responded to the first time. But you need to start paying attention. You denial & empty confidence in what's happening will only get you so far.

View on HN

Considering my former co founder works at OpenAI and we spent years working on and talking about related ML with and around some of the best people in the world, considering I’ve implemented language models multiple times myself, please try and be more specific on where I’m wrong.

I know many people who work in ML share this concern - people thinking it’s smart, it’s more than it is.

It’s very useful no doubt, but your comment is an example of extrapolation from maladjusted priors.

Your example proves my point, a very uninteresting looking game. I don’t even doubt GPT will find all sorts of interesting stuff by giving averaged out things. There’s plenty of gaps in the averages humans have missed. But nothing new, and until you can show that let’s just call it what it is.

Your goalpost moved from "nothing novel" to "it's novel, but it's not interesting to me". I'll let not the years ahead, but literally the DAYS ahead prove you wrong. Because things are changing quite rapidly lately.

I've seen ChatGPT draw logical conclusions from (most-likely) previously unseen axioms. I've seen it invent ideas that (most-likely) were not in his training set. I've seen it fail too, but when it succeeds I can not substantially distinguish it from the novelty or logical reasoning of a human.

On niche subjects, it fails often. But you can get it to correct itself by making it think logically by spelling out axioms, which seems to fit your definition of novelty.

All the arguments I've read about it not being intelligent seem to stem from some form of essentialism. Right now it's dumb intelligent but I think giving it an adjustable recursive depth, similarly to what BulgarianIdiot suggests, will go a long way to improving it given its base capabilities.

Indeed. And telling it to spell out the steps in math solution is essentially (ab)using the chat log as the memory for an iteration loop.

This demonstrates then when you instruct it "break down the problem and solve it iteratively" it comes out with superior solutions. More accurate, more novel. Ergo, then if it was allowed to iterate in this way internally, it'd produce better answers by default.

This thread sums up the same idea: https://twitter.com/ItIsFinch/status/1634730897520951296

Yeah, I think it's going to be a promising area of research. It's known as Chain-Of-Thought (CoT) in the literature and there are a few papers already out:

https://openreview.net/forum?id=_VjQlMeSB_J

https://openreview.net/forum?id=5NTt8GFjUHkr

https://openreview.net/forum?id=qFVVBzXxR2V

I'm also looking forward to explicit long-term memory mechanisms that don't rely on the main architecture's weights nor on CoT.

There's no memory or iteration log. You are just priming the (more or less deterministic even...) predictor with some history. You are doing a lot of work yourself, setting a factually correct context (and this has severe limits...).

As for "we just need a way to include iterations": isn't this what attention is supposed to do (kind of dynamically updating weights). The usual way to really update weights are variations on gradient descent. Can you link a paper outlining how we integrate your proposal into the current framework (which took around 20years to mature)? Otherwise your statements are Sci-Fi (at the flying cars-level).

Oh so there's no memory, just recallable history.

Oh so it's not learning by example, it's just primed by a sample.

Oh so there's no iteration of thought, just a loop of prediction.

OK.

Also, the papers you asked about: https://news.ycombinator.com/item?id=35115563

BTW, people are already implementing recursive/iterative queries on the ChatGPT API, and getting promising results.

The langchain project is an example of the iterative queries approach. It comes with constructs for working memory, factual lookup / calculation agents, etc.

https://github.com/hwchase17/langchain

The general (non-technical) guideline is that the LLMs can "answer" anything you just gave them the answer for. So you give it a problem, ask it how to solve it, tell it to use that method and explain the data it needs, give it that data, and then show it everything at once: "With this data you requested and summarized, use this technique to answer this question".

Are you on other social media BTW? Thx.

Afraid not, only here. You can at me at scrap.titanium@fastmail.com though.

> I've seen it invent ideas that (most-likely) were not in his training set.

How can you possibly know what's in its training set? Its training set is so large that any "guess" about what is or is not in it is a fool's errand. Given that we know how the technology works, there's no reason to claim emergent behavior when Occam's razor would say the items are likely in the training set.

It's not a fools errand, you can't know definitively but you can have a level of confidence.

It's quite easy to invent a sentence that's never been uttered before, it's also fairly easy to hit the limits of the internet's knowledge in pretty much any complex discipline that's not software engineering.

And it's apparent when you speak to ChatGPT, he will hallucinate a lot to compensate for niche topics it doesn't know much about.

Occam's razor wouldn't point to ChatGPT having omniscient knowledge of all things ever thought of and that will ever be invented.

This kind of stuff clearly shows that GPT can generalize concepts:

https://twitter.com/emollick/status/1626084142239649792

And this kind of stuff shows that it clearly cannot:

https://acoup.blog/2023/02/17/collections-on-chatgpt/

> If you know nothing about either book, this answer almost sounds useful (it isn’t). Now this is a trivial research task; simply typing ‘the limits of empire review’ into Google and then clicking on the very first non-paywalled result (this review of the book by David Potter from 1990) and reading the first paragraph makes almost immediately clear the correct answer is that Isaac’s book is an intentional and explicit rebuttal of Luttwak’s book, or as Potter puts it, “Ben Isaac’s The Limits of Empire offers a new and formidable challenge to Luttwack.”

> A human being who understands the words and what they mean could immediately answer the question, but ChatGPT which doesn’t, cannot: it can only BS around the answer by describing both books and then lamely saying they “intersect in some ways.”

> The information ChatGPT needed was clearly in its training materials (or it wouldn’t have a description of either book to make a lossy copy of), but it lacks the capacity to understand that information as information (rather than as a statistically correlated sequence of words). Consequently it cannot draw the right conclusion and so talks around the question in a convincing, but erronous way.

That appears to show that it can't answer some obscure trivia question to the standards of the blog post author. Nothing to do with generalizing concepts.

It's not an obscure trivia question. It's a question that anyone with an understanding of the material in question should be able to answer, with something vastly better than a superficial 'the two books are similar, but, like, different and talk about different stuff'.

It immediately cuts to the heart of the matter - an LLM does not understand things. It does not know anything but which word follows another. It can't do anything but regurgitate things it has read in an incredibly lossy manner.

But it's a neat party trick that seems to fool a lot of people into thinking that there's something legitimately useful, here.

Large language models are not search engines, even though a lot of people seem determined to treat them as such.

I don't think that blog post's excercise is a particularly useful demonstration of anything.

> We're already experimenting with variations that can iterate internally, and therefore think deeper, using still their existing knowledge. And the results change dramatically.

This is still just deterministic except for a bit of noise to vary the output. There is no "thinking" going on. What you get is still just the most statistically likely output based on the training data.

You're also deterministic. Except for a bit of noise. This is less of a slam dunk argument than many think it is.

For reference, I enjoyed playing sumplete. Master level passed the time nicely (not the full 20 hours, but I did come back to it!).

Mechanics work well for an AI generated game. No idea if it's novel or not, but it's a good take.

I think people also often figure out exactly what they think mid-sentence, by way of actually putting it into words. LLMs can iterate on their ideas as they generate the text.

> You're also not right that GPT can't produce anything novel. Here's a tiny, modest example

Actually it turned out it didn't invent any of the games in that article. You can just Google them and find out.[0][1]

Reading is not believing.

0: https://www.digitaltrends.com/gaming/sumplete-chatgpt-ai-gam...

1: https://www.novelgames.com/en/labyrinthsudoku/

> I won't waste my time, as I see you didn't read what you responded to the first time. But you need to start paying attention. You denial & empty confidence in what's happening will only get you so far.

Maybe he didn't read what you wrote, but you didn't even bother Googling the claims in the article you posted.

I wonder how ChatGTP reinvented its own game? That indicates it's being retrained very frequently on crawl data which I wasn't aware was happening - certainly not with a cycle time in the weeks!

It didn't. The article is nonsense. He probably primed it beforehand by talking about Sudoku, or it may have just chanced onto something similar. When I ask it to code Sumplete, it creates a "math-based game in which players have to solve a sequence of simple addition problems within a limited time to progress to the next level."

It says:

> “Can you code a game called Sumplete?” Despite protesting about its inability to make games earlier, ChatGPT immediately started spitting out fully formed HTML, Javascript, and CSS code. Sure enough, it had once again created a version of Tait’s game – only this time, we never discussed what it actually was beforehand.

I guess "a version of Tait's game" leaves some wriggle room about exactly what it did.

The article seems to leave for itself wiggle room in several such places, suggesting the author had an agenda. Likely ideological rejection of AI as intelligence.

There is no evidence at all that it copied the smartphone game. No one has even been able to show a textual description of its rules online that could be copied. And modifying the rules of Sudoko to create a variation is well within ChatGPT's known abilities. You can easily test this stuff yourself.

It's quite bizarre to watch this discrimination against AI forming, where we make these increasingly more intelligent entities, and now everyone is out to prove they're stupid, and by definition they'll always be stupid, because only humans can be smart.

I don't want or need to anthropomorphize AI (yet?), but this reaction by us is a tad too similar to rather uncomfortable parts of our history.

Did you read carefully?

Your second link is a game different than Sumplete's rules, so not sure why you linked it.

You have one source, the Digital Trends article. It speaks about a mobile game being similar. The link here is at best circumstantial. The app page doesn't describe the rules of the game. Instead it says this:

"It is very similar to classical game of Kakuro"

I checked Kakuro, and it has similar, but distinctly DIFFERENT rules, as well. So, what's the theory here? GPT downloaded the game and ran it on its phone? No. Maybe the description of this game is somewhere. Maybe. But I couldn't find it. Where is it?

The excuse "it's somewhere, so it didn't invent it" will always be used for why an AI can't produce a "novel idea". While the reality of novel ideas is very simple: it's like conservation of matter & energy. You can never make something or destroy it. You can only rearrange it. New ideas are a rearrangement of other existing ideas. It's the case for everything we ever made. And everything we ourselves are.

This is a discussion in the art world that you'd hear a lot, before stable diffusion came out. The whole idea of "steal like an artist".

It's one of those questions about the nature of us that's on the level of "is there a ghost in the machine"; "do we have actually novel ideas?".

I'm kind of leaning the way you are; I don't think we do. What seems to be happening that we _call_ novelty is just an advanced form of recursive synthesis - we take information in and remix it. But we're capable of levels of abstraction, so we can remix it really well.

I think that getting the abstraction part into these models might just be as simple as wiring them together with a meta-model. A network responsible for identifying similarity between outputs of different types, or one that serves as a connective layer, distributing tokens through other networks and then synthesizing the results.

What I'm almost certain of, is that the important changes are going to be architectural.

The important changes will be architectural, yes.

Three things I'm noticing:

1. We're reinventing structured programming in AI in fast forward. First it's a plain Markov chain. Then it's an "attention" directed acyclic graph. Then we realize we need loops. Then we realize we need to jump to different points in the loop. Then we realize it's useful to recursively call yourself or parts of yourself as a subroutine, parametrized with specific input. Etc.

2. Even before we fully realize this framework of thought into a model, I'm almost sure the model EVOLVES some of these structures during training. In the form of crude unrolled loops etc. Simply because it's inevitable for processing certain types of input data.

3. In order to preserve pragmatic outcomes, I'd bet the future is not one giant monolithic model for AI, but many medium-sized models, communicating in a meta network, like meta neurons, sending meta (high-level) messages to each other.

Essentially, we need to make neural networks more like a fractal. I have this rule of thumb that always works somehow: "no concept definition is complete, until it's made recursive". Neural networks will get there.

Reply on news.ycombinator.com