Hacker News Reader

Comment by Donckele | Hacker News Reader

1) 1.8% is still massive 2) 1.8% is still massive 3) what?

1. No it's not lol. If the model was only trained on that much data, it wouldn't be anywhere near as good in french. 1.8% is only enough here because it trained on other languages as well.

GPT-3 is also fluent in languages with less training data.

3. LLMs trained on code score noticeably higher on reasoning benchmarks

lol?

1.8% does look like a small number but imagine (i know its hard in this day and age with 4TB finger nail usb sticks) a physical library holding good old fashioned paper made artifacts and what does 1.8% of that looks like?

It’s also fluent (though not at a native level) in, say, Finnish, and can write reasonable poems and whatever, even though Finnish text probably comprises something like 0.01% of the corpus.

Norwegian as well. Though given Norwegian is also a Germanic language like English, Finnish is a lot more impressive. I managed to get it trip up on one single gendered article in Norwegian, while asking about Sweden, in a way very similar to how a human switching between Norwegian and Swedish might (it would be valid Swedish, and was followed by a Swedish word that's also valid in Norwegian but with another gender), but for the most part it writes Norwegian better than a lot of native speakers I've known would write (but not better than how they'd speak).

EDIT: I got it to "translate" a test sentence from Norwegian to "Riksmål", an old-fashioned artificial Norwegian dialect that seeks to stay closer to very conservative old Norwegian (basically a lot closer to Danish) and it managed that just fine too, and there's very little material that you'd be able to tell explicitly is Riksmål as most of the time it'd just be described as Norwegian unless the writer is trying to make a point.

Since that worked, I tried Nynorsk ("New Norwegian") which is used by something like 20% of the population, and it did fine.

So I tried a local dialect from Eastern Norway, that is predominantly spoken and only occasionally written and it got even that pretty much how I'd have said it myself, after a lecture on how it varies greatly within the region (which is true)

The differences there are subtle, and frankly a lot of native speakers would get one or more of these wrong.

EDIT2: Last try at stumping it with Norwegian: "AKP-dialect". AKP was a Maoist political party of a few hundred members whose members were largely intellectuals. Many of them adopted a "faux workers dialect". This had no formal name, so it takes some understanding of Norwegian 1970's and 80's politics to even guess what it is. It understood what I meant, but didn't understood what spesifikk language-features I was referring to. Then again, if I asked random Norwegians about that most would need an explanation too. When I mentioned a couple of people who wrote like that it produced a plausible version (but clearly more inspired by the writing style of one of them - Jon Michelet)

Reply on news.ycombinator.com