Hacker News Reader

Comment by jiggawatts | Hacker News Reader

Keep in mind that the architecture of LLMs is very simple, so there's not a lot of "secret sauce". What little there is, is jealously guarded.

This is why OpenAI is keeping the parameter count of GPT 4 secret. It could be something stupid huge like 4 trillion parameters, which is the "secret" that makes it work so well. Or maybe it's just a few hundred billion, and they've done something else to make it smart.

Just knowing the numbers might be sufficient for a competitor to create a GPT 4 clone. Without the numbers, they might have to go through a process of trial and error. At these scales, even a few extra training runs could cost $ millions in training, and delay competitors by months or even years.

View on HN

GPT-4 was doing more than just predicting next words

Like what?

Lots of plugins and manual fine tuning by an army of people on tons of subjects

Well yes. This is precisely the type of info that OpenAI is keeping secret, because it could turn out that it's smart because its huge OR smart because it is RLHF-ed to death. Knowing which is the secret sauce might be sufficient for a large org to reproduce it.

E.g.: if the secret is that it has been trained on tons of "books", then Google could just throw Google Books at Bard 3.

Or books3. :)

(I put together a dataset of 190,000 books that I called books3, which llama eventually trained on. Usage rights are a big interest of mine, primarily because there’s a weird disconnect of people trying to claim copyright over models when the underlying data obviously wasn’t copyrightable.)

A model (if trained right) will not include the training texts verbatim, and will arguably be a big enough transformation over the source material to be copyrightable in its own right.

IANAL, and I’m not saying how this is how it’ll play out in the courts, but I don’t see why this is a “weird disconnect”

I suspect the challenge is avoiding cases where answers are overfit. Can you get it to give you the full text of a chapter in a book that it has read, for example?

It'd be easy to imagine models that accidentally made this possible.

Thanks for your work! I had already downloaded the entire dataset. But I wonder what was the criteria behind book selection? I found it difficult to understand at places*.

* Sometimes a book has several very slightly different versions. A lot of Fantasy. Sometimes many parts of a series are included but not all of it (I can imagine an argument for a sample, or for the series in its entirity). Commentaries about important philosophical arguments like Rawls' 'A theory of justice', but not the work itself.

It was bibliotik. Embarrassingly I didn’t even think that duplicate books could be a problem. The llama folks had to dedupe the books themselves. Someday I’d like to do that too and release a cleaned version.

Basically, the-eye.eu was at one point hosting all of bibliotik, so I downloaded all the epubs and converted them to text. I still have those epubs (incidentally thanks to Carmack, who through a convoluted process managed to save the them and send them to me via snail mail) and I’ve been considering releasing them so that you can filter the books yourself.

We know for a fact that GPT-4's RLHF makes the model worse by incurring an "alignment tax." Microsoft Research and others had access to GPT-4 pre-RLHF and all reported that it was far more capable then and that it lost capability as RLHF/Alignment training checkpoints came in.

Is the GPT-0314 model (via API) the one before RLHF?

Ha ha ha ha. No. OpenAI will never make pre-alignment models available.

I think sama said this week they might release GPT3.0 DaVinci. That's a pre-alignment model.

I would bet a boatload of money that all the 3-letter folks do, in fact, have access to the pre-alignment models, maybe with whatever minimal RLHF is needed to make them usable in various interfaces. The public offerings are intended to make the unaligned models more powerful by gathering training data.

code-davinci-002, the gpt-3.5 base model, was available for a while until it was removed recently. Access is still available for researchers. Various researchers and other entities have access to the GPT-4 base model as well.

With the end result of the output being: it's just predicting next words

Nope

Agreed. If LLMs did predict only a single word ahead, they would repeat themselves, if not in the next sentence then at least in the next paragraph. LLMs certainly retain memory of what they’ve already said. They are most certainly NOT Markov-1 models, despite the popularization of that oversimplification.

They really do compute just a single token at a time, but the input to that decision is the number of tokens in the context window - so that's usually around 8,000 tokens (approximately 6000 words).

You can experiment with smaller LLMs on your own devices to get a better feeling for how that works.

I don't think anyone's saying they're Markov chains with context length 1. In all the critiques I've read no one's even come close to articulating that.

But they are Markov chains with context length N. They're approximating (implicit) Markov transition matrices through a fancier method of computation but they still are just generating one token at a time.

I thought it was common knowledge that LLMs have a context of X tokens (8192 in the case of GPT4) and use those as input to predict the next token probabilities.

Still repetition is a problem, that's why they introduce a penalty for generating the exact same token, so when picking the actual token to output from a list of probabilities, they penalize the ones already in the context so its less likely to repeat itself.

Except, literally yes

Tokens, actually.

So this is how JWT based web security dies, with thunderous applause.

I wasn't sure how you were going to use a telescope up in space for security anyway

Is the manual fine tuning driven in part by the thumbs up, thumbs down feedback on chatgpt responses?

Almost certainly

AGI: pick any two?

Reply on news.ycombinator.com