Comment by qayxc | Hacker News Reader

Comment by qayxc | Hacker News Reader

Parent Full thread

> do you have any idea of what things machine translation still consistently gets wrong?

My biggest pet peeve with MT is the incredibly brain-dead choice of using English as an intermediate language.

This effectively prevents MT from ever becoming useful for any A→B translation where neither A nor B are English.

Example: put "пружи́на" into Google translate (or Bing translate, doesn't matter)and translate to German. You'll get "Frühling" as the suggested translation.

This is due to the fact that internally "пружи́на" is translated to English first, resulting in "spring". This is then translated to German with the statistically highest probability being "Frühling", even though "Feder" would be the only correct match ("Frühling" would be "весна" in Russian).

If you now say "But wait! This should clear itself up once you add some context, right?" you'd be wrong.

Try "Die Sprungfeder kam gerade rechtzeitig." ("The coil spring arrived just in time,") and you'll get "Весна пришла как раз вовремя." ("Springtime came just in time").

It's ridiculous!

EDIT: happens with DeepL as well; only Yandex gets it right (presumably because they actually use an actual German/Russian-model.

> This is due to the fact that internally "пружи́на" is translated to English first, resulting in "spring"

Do you have a source that that is actually what is happening behind the scenes? I work in this field and would be surprised if that is what Google is doing. My guess is this phenomenon is more due to a model that was trained on an Russian->English corpora first and then trained on Russian->German due to low resource, or something like that.

Someone like Yandex is going to have better translations because (shocker) they have larger russian corpora than google.

> Do you have a source that that is actually what is happening behind the scenes?

Indirectly yes, I do: https://codesachin.wordpress.com/2017/01/18/understanding-th...

> What this basically means is: If during training you provide it examples of English->Japanese & English->Korean translations, GNMT automatically does Japanese->Korean reasonably well! In fact, this is the biggest achievement of GNMT as a project.

It's exactly what Google have been doing for years now: train on X-to-English and English-to-Y only to get X-to-Y for free. Since the intermediate language only ever sees English as a source or target language, ambiguities like "spring" literally get lost in translation.

Just gave that a read.

1. It is pretty out of date (ML is a fast moving field) - I doubt Google is using LSTMs for translation in 2020.

> Since the intermediate language only ever sees English as a source or target language, ambiguities like "spring" literally get lost in translation.

2. This article is trying to dumb down what Google is doing - but you're right that this is why ambiguities get lost in translation, due to pre-training on a different language pair.

That said, there isn't literally a process of "translate into english" and then "translate english into german". These models are trained on Russian-German corpora, but because there is little resources for that, they are supplementing with Russian-English and English-German.

> It is pretty out of date (ML is a fast moving field) - I doubt Google is using LSTMs for translation in 2020.

Sure, but that doesn't change the fact the training data is focused on English-to-X and X-to-English corpora. The underlying architecture of the model is just an implementation detail that doesn't really affect this as demonstrated by my example.

> These models are trained on Russian-German corpora, but because there is little resources for that, they are supplementing with Russian-English and English-German.

This is exactly what I'd argue isn't the case at all. Otherwise words that have a direct 1:1 translation wouldn't be mistranslated and companies like Yandex wouldn't be able to deliver so much better results.

German-Russian isn't low-resource at all, given 95M and 150M native speakers respectively and a close history for the past 150 years. [edit]The rich cultural history of both countries resulting in a vast library of literature, theatre plays, news publications, films and the general cultural relevance of both languages is even more important.[/edit] It's simply (quite comprehensible) bias towards English for research taking place in the USA and the fact that it's much easier to compile English-to-X and X-to-English corpora in a predominantly English-speaking country.

There are tons of translated books, films, news paper articles, scientific papers, etc. available for Russian-German and Yandex, being a Russian company, naturally has no problem compiling a Russian-German corpus (since they're not biased towards English).

Whatever the actual reason is, the results feel like English is used as an intermediate.

This also happens between e.g. Spanish and Bulgarian.

It's not brain-dead, it's a result of limited training data.

For N languages you need to train N models if you go through English as an intermediate language, but N^2 if you don't use an intermediate language model. OK, computer time is cheap these days, but more importantly, for each language pair you need D high-quality documents translated in each of your two languages.

There are far fewer pieces of writing that exist in both German and Russian than in both German and English or both Russian and English.

There is cool-looking research on zero-shot language translation where the model learns its own internal "intermediate language". But you can be sure that if it produced reliably better results with the data available, the likes of Google would already be using it.

> It's not brain-dead, it's a result of limited training data.

That's quite literally a cheap excuse. It'snot lack of training data (remember, you still need translations in the target language anyway!), it's a result of Anglo-Saxon cultural hegemony.

> There are far fewer pieces of writing that exist in both German and Russian than in both German and English or both Russian and English.

Right. And that's why Russia-based Yandex gets excellent results for German/Russian translation, I see...

> But you can be sure that if it produced reliably better results with the data available, the likes of Google would already be using it.

Nah, you're being way too optimistic there. Google is a business and as such, they don't chase SOTA outside of research (which doubles as advertisement). Good enough is the name of the game and it's simply infinitely cheaper to just train and maintain N-models (i.e. English/X and X/English) than to do the same for all combinations of X/Y-pairs (which would be N² models).

So as long as there is no incentive to do better, they simply won't, because they rely on the fact that English is the de-facto Lingua Franca and people translate to and from English more often than directly between other languages.

This would be an actual chance for companies from other countries (especially European ones) to step in and do exactly that, but again - "good enough" and reliance on English stops that from happening anytime soon. It's a missed opportunity especially with low-resource languages as demonstrated by the recent "Scottish" Wikipedia fiasco...

Reply on news.ycombinator.com