Hacker News Reader

Comment by namaria | Hacker News Reader

> I doubt any biological system can ever learn all the information gpt4 has learned

The fact that human brains can become competent at language with training datasets with a size of a tiny fraction of common crawl points to how much more efficient they are

View on HN

Until you realize that ChatGPT is also fluent in 20 other languages by including minimal data into the training dataset.

Minimal as in the amount a person needs to learn the language or minimal as in tiny if compared to English in common crawl but still orders of magnitude more than a person goes through for infancy language acquisition?

I think you know the difference.

It’s anecdotal, but here [1] is an example of AI learning to translate a new language faster than a human could learn to do it, from a similar amount of data, using Claude 3 Opus.

Translating a language isn’t quite the same as understanding it, but this is still impressively data-efficient.

[1] https://twitter.com/hahahahohohe/status/1765088860592394250

Did you read the "added context"?

"The author has later responded and apologized a flawed / biased methodology which led them to believe that Opus has no prior knowledge of Circassian (i.e. wasn't trained on it)

Apparently it was, and is able to speak Circassian just not perfectly"

I didn't see that! Thanks!

Bro you are proven wrong. Baby FOV camera footage as the training dataset of LLM, the result is not bad.

https://www.nyu.edu/about/news-publications/news/2024/februa...

The result is bad, that is just an image recognition model we know those can be trained with relatively little words and just a bunch of images, the model didn't learn grammar or sentences or relationships like a baby would. The only models we have that can do grammar and sentences and relationships are LLMs with massive amounts of data.

Reply on news.ycombinator.com