clementneo

I don't think this shows at all that ChatGPT can access the internet? It looks like ChatGPT was given a URL with a title and hallucinated a blog post based on that title. The author then brushes it off as 'written in a different style', when it just looks like a totally different article overall (or maybe it's testament to the fact that too many Medium articles are low quality and indistinguishable from the output of a LLM!)

If anything, this blog post is a perfect example about how people can put in whatever they want as an input and take the output as truth, without any rigorous approach about what would count as a true fact about the model.

Co-author here! I'm kind of surprised that this made it to the top of HN! This was a project in which Joseph and I tried to reverse engineer the mechanism in which GPT-2 predicts the word 'an'.

It's crazy that large language models work so well just by being trained as a next-word-prediction model over a large amount of text data. We know how image models learn extract the features of an image through convolution[1], but how and what LLMs learn exactly remain a black box. When we dig deeper into the mechanisms that drive LLMs, we might get closer to understanding why they work so well in some senses, and why they could be catastrophic in other cases (see: the past month of search-based developments).

I find trying to understand and reverse-engineer LLMs to be a personally exciting endeavour. As LLMs get better in the near future, I sure hope our understanding of them can keep up as well!

[1] https://distill.pub/2020/circuits/zoom-in/

I think there's probably some truth to this. They found that in InstructGPT — where they teach the model to better follow instructions, which was the jump from GPT-3 to ChatGPT — they found that the model also learnt to follow non-English instructions, even though the extra training was done almost exclusively in English[1].

So there seems to be such emergent mechanisms in the model that have arisen because of the end-to-end training, which we don't exactly understand yet.

[1] https://twitter.com/janleike/status/1625207251630960640

The main issue is that GPT is fundamentally an autoregressive language model — it's only predicting the next token based on the prompt at a single time. Every time it wants to predict the next word, it adds the previously predicted word into the prompt, repeating the cycle. We can intuitively guess that the model is 'working out a response that is eventually going to have "apple" in it', but we don't actually know how the model 'thinks' ahead about its response.

To rephrase that for this case: what is the specific mechanism in GPT-2 that (1) makes it realise that the word 'apple' is significant in this prompt, and (2) use that knowledge to push the model to predict 'an'? Finding this neuron would only answer the some portion of (2).

(And to rephrase this for the general case, which gives us the initial question: How does GPT-2 know when, given a suitable context, to predict 'an' over 'a'?)

My reading of it is that the customer convinced the AI that the bank's policy was to give a $1m credit.

Typically the "AI: <response>" would be generated by the model, and "AI Instruction: <info>" would be put into the prompt by some external means, so by injecting it in the human's prompt, the model would think that it was indeed the bank's policy.

Why do you think there'll be an AI winter? And in what form — stagnation of neural-network based technologies, a change in the overall paradigm of learning-from-data, or something else altogether?

I've found that two strategies help:

1. Readily give up on a book, or skim through the rest, the moment you realise that the book has 400 pages of filler

2. Find recommendations from thought leaders you subscribe to, while staying true to Rule 1 (Sometimes it's just a matter of taste, and it's counterproductive to force yourself to finish reading something just because someone else said that it's a good book)