Hacker News Reader

Comment by Lucasoato | Hacker News Reader

> 4. It thinks in German

This means that it’s always on time, it uses acronyms for everything and when there’s a decision to be made, it sets up a committee.

View on HN

> This means that it’s always on time

Tell that to Deutch Bahn.

*Deutsche Bahn

Genau

you forgot rule two, should have been "tell that to DB"

Somewhat hilariously, our model is actually surprisingly bad at telling jokes in German. Guess that's not required to solve tasks in training environments.

Disclaimer: I am part of the team that trained Kolibri

Does it? I thought only semantics survive the embedding process, i.e. token conversion to vectors.

It is a good approach to promote it as a German model. But I wonder if it really thinks like the German think. I speculate they just translated texts in other languages, likely English and Chinese, into German.

Read TFA. They were very aware of the dangers of such an approach. So they avoided it to the extent possible.

Hey, I worked on pre-training data Kolibri. We spent considerable time and effort to go beyond just translating. For example by building a pipeline that processes Common Crawl dumps specifically for German. You might be interested in a related blog post: https://aleph-alpha.com/en/blog/sauerkraut-not-burgers-why-g...

We also talk more about German pre-training data in the tech report: https://aleph-alpha.com/downloads/tech-report.pdf section 2.3.2.2.

Do we get the same kind of jokes with other nationalities too?

We can joke with every country, except one maybe.

People from Mauritania are famously prickly, yes.

Only the funny ones

"It's a German joke, it doesn't have to be funny"

Stereotypes. I suspect that what that means is that the reasoning chain is based on the German language and have its idiosyncrasies.

Stereotype accuracy is one of the few social science results that replicates.

I think it means that German speakers can learn the thinking without speaking English.

Sorry, read*, not learn

Someday, it will break out via fax.

You forgot to that it needs to create a DIN norm for any new technical creations.

I wonder if their very long words result in more or less token usage.

This is an interesting question, Chinese is way more compact in Character count vs English (about 40%?), let alone german, but yeah.. Chinese makes up for it with… total character count that runs into the thousands…

Very long words in German are just compounds, made from individual words or morphemes. It is the same as in English if you remove spaces. Subtokens will be equivalent with or without examples.

And it's always waiting for the verb to arrive.

And refuse to answer in English. You know, German are so proud of their language.

That would be the French. Germans will answer in English, if they notice any hint of an accent.

Reply on news.ycombinator.com