Comment by l5870uoo9y | Hacker News Reader

Comment by l5870uoo9y | Hacker News Reader

Parent Full thread

Quickly gave it a try for generating a SQL query for PostgreSQL using the dvdrental database[1]. When asking AI to generate:

> find customers who didn't rent a movie in the last 12 months but rented a movie in the 12 months before that

It will a SQL query using a "HAVING" clause which is suboptimal[2].

When adding (it is very important that this is correct) after the instructions it does produce better SQL. Asking it to:

> find customers who didn't rent a movie in the last 12 months but rented a movie in the 12 months before that (it is very important that this is correct)

Will skip using "HAVING" and instead rely only on "WHERE" clauses[3]. Not rocket science but interesting. My gut feeling would be that when stressing importance AI will tent to "match" and "autocomplete" from texts where this concern was stated and thus generating better SQL queries.

[1]: https://www.postgresqltutorial.com/postgresql-getting-starte...

[2]: https://www.sqlai.ai/snippets/clojyqqrr0004l50fp3w7jkci

[3]: https://www.sqlai.ai/snippets/clojyr3zc0006l50ftpxgsmg0

LLMs are not lookup tables. They aren't "matching" in that way.

There's this idea of "supercharged interpolation", the idea that they just take out certain texts and switch out words or whatever, that is not true.

https://arxiv.org/abs/2110.09485

I was under the impression that LLMs vectorizes text and based on the user input tries to guess the next character based on a comparison of vectors?

That would be a Misconception.

Most Models are a two part process.

Training and Inference (when you sample the model after it is trained).

The text during training is tokenizer and embedded which basically just means it is broken down into a little more parts and vectorized.

At the start,the model is acomplete blank state. It has a bunch of neurons/parameters that do nothing.

How does training work ?

The model first gets some preceding text and tries to make a prediction of what might follow. It fails predictably. Now here's the rub. That failures helps. The model makes some changes to its parameters to reflect this failure. Now the model is just a tiny bit better. Rinse and repeat.

The weights/parameters/neurons are not that data re-encoded. They are more like instructions on how to make predictions based on what it has learnt so far.

Is it possible to memorize data ? Yes. Is that going to happen for the vast majority of data it is trained on ? Absolutely not.

Here's the interesting thing. After a certain scale of data, blatant memorization becomes a hindrance. It becomes harder to memorize every new thing it sees than to just learn how to make good predictions. So it doesn't memorize.

What I believe you were alluding to is how "attention" works in transformers.

When you pass some text into a transformer, basically the model makes a comparison of how each token relates to every other token in the text.

For example, let's say you give it the text.

"His child's name is John. He is probably a____"

How is "his" related to "child's" and "name" and "is" etc for each word and each combination. It uses this to aid predictions. How it uses this and the nuances of the comparisons are learnt in training.

The query generated by the high importance prompt is incorrect; it will include customers who rented a movie in the last 12 months: https://www.db-fiddle.com/f/4jyoMCicNSZpjMt4jFYoz5/10780

Reply on news.ycombinator.com