Hacker News Reader

Comment by anonylizard | Hacker News Reader

Of course. The larger the model is (GPT3.5+), the more languages it can hold in memory, so they are naturally multilingual.

But small models (Llama) need dedicated training to be good at another language.

Its similar to how high verbal IQ people can easily master multiple languages, but normal people are only good at their main language.

View on HN

> Its similar to how high verbal IQ people can easily master multiple languages, but normal people are only good at their main language.

Let me guess: You are American?

That is very much not true. Go to the Netherlands, for example. Pretty much everybody under 50 speaks very good English. You don’t have to have a high verbal IQ, or be somehow abnormal to become good at a non-native language.

I guess the fact that Dutch and English are pretty closely related helps a lot here. There are definitely people who have trouble learning a language that is from a completely different family (it doesn't even have to be Chinese, try Finnish or Hungarian).

> Its similar to how high verbal IQ people can easily master multiple languages, but normal people are only good at their main language.

How? Are you implying "high verbal IQ people" have bigger "models" in their heads somehow?

That quote sounds like made up nonsense. I'll wait some evidence and ignore it in the meantime.

There's a lot that goes into verbal intelligence, and we don't really know all of the what and how. It'd be cool if we did, but for now we can only really draw some correlations to physical areas of the brain, which are nevertheless fluid and unique in each individual, and some genetic links. Even then it isn't enough to account for half the stuff we observe humans doing – in Europe learning between 2-4 languages as a child is considered pretty much entirely normal.

Yeah as far as I understand NNs do mimic neurons in a simplistic way but any parallel between them and human brains beyond that is bound to fail because we don't know how human brains work that well. We do not learn by scouring millions of tokens. And regular children can learn a language faster than a high IQ adult.

I think learning multiple languages is normal for children in most places, especially if different social contexts require different languages. Lots of North Africans speak Arabic, French and English for example and some of them speak berber languages on top of that.

It’s also heavily dictated by cultural factors. in northern European countries it’s quite normal to know at least English next to your native language. So much so that in finland tourists seem to think that everybody knows English and foreigners who try to learn finnish, tend to complain that Finns switch to English at every opportunity.

Cultural factors are one thing, the other is just that there are only ~5.8 million native Finnish speakers, so a lot of books, movies, music, etc. are not available in the language for purely economic reasons. There are over 90 million German native speakers, and TV shows and movies are dubbed, books translated, etc., so there is less of a need to have high proficiency in English (similar to Spanish-speaking countries).

You'd be surprised, honestly. What matters is also comprehensible input size, so long works like LOTR (which has an excellent Finnish translation by Kersti Juva for anyone looking to find works to read) are a good choice. Similarly, there's a lot of Finnish literature in Finnish which Finnish people spend time reading, some of which is translated to many, many languages. The Egyptian by Mika Waltari is one of those classics. Not nearly everyone here knows English well. TV shows and movies are usually subtitled as well, save for kids' shows. Finland does however have three mandatory languages in the basic schooling, which are your native language, the other native language, and then English. A fourth one is not uncommon either, so we have a lot of German, Spanish, Russian, Japanese etc. speakers here. Hell, jag kan neljää языки.

> Its similar to how high verbal IQ people can easily master multiple languages, but normal people are only good at their main language.

Normal people can be good at multiple languages; people living in large nations whose own language also effectively the global libgua franca often don’t because there is little marginal benefit for them to do so, bot because of verbal IQ. People of similar “verbal IQ” but living in an area where a few hours drive can put them in places with three or four different prinary languages will often be functional in several languages.

That last paragraph is a bold claim, my friend.

Not really? "Verbal IQ" has a specific meaning, one related to language learning.

Fair enough, though if meant that way it’s not that meaningful as comparison(?). It loosely translates as: “People who are good with languages are good with languages”.

Historically, and even today on a global scale, monolingualism is the exception. In the absence of large state actors (i.e. through most of human history) languages diversify much more than we are aware of today, and this requires many more people to be familiar with multiple languages.

I’ve heard (credible-sounding) gossip that most folks hit a 1.5Tw 16-way mix on GPT-4. Is that like, consensus these days?

A couple of months ago I got 3.5-turbo to start talking about “the details” but that is about as reliable as anything it says. It was claiming like, 200Gw.

Are solid numbers available?

> A couple of months ago I got 3.5-turbo to start talking about “the details” but that is about as reliable as anything it says.

Less reliable, probably. Very unlikely this information is in the training set.

Reply on news.ycombinator.com