Bing Translates “Daesh” as “Saudi Arabia”, Angers Entire Kingdom

Machine translation systems are trained on a large parallel corpus of sentences of variable length. The models are extremely opaque (a complex system whose core is typically deep neural network). They are trained to take an arbitrary-length sequence of symbols as input, and produce an arbitrary-length sequence as output.

There's basically no human in the loop. Data goes in, model comes out, it's tested for accuracy (using a metric called BLEU) on a test set, and if it does well on the test set, it is shipped. These systems are immensely complex, and it'd be quite silly to manually check for every possible error case.

I seriously hope Bing translation service is more advanced than a static key/value word map! Probable additions would be a grammar parser, and a NN-based translator, with a training set constantly updated.

Translation services are generally trained on parallel text, where we believe a given sentence is the translation of another sentence.

You don't specify which word is the translation of which word. That would take a lot of human effort, and wouldn't even have a correct answer in some cases. You just throw a lot of parallel sentences at a machine learning algorithm and let it infer how to translate the words, somewhere inside an inscrutable model.

My guess, then, is that "Daesh" is a relatively new word that is not represented very well in their parallel corpus of Arabic text.

Suppose it has just one sentence of parallel text that mentions Daesh, and that sentence also mentions Saudi Arabia, but the words appear in a different order in English and Arabic. It doesn't have enough evidence to figure out that the word order is different, and it ends up aligning Arabic "Daesh" with English "Saudi Arabia".

Now, Google Translate at least has a fallback on a dictionary, and you can see this when you enter a single word. But maybe Bing's dictionary isn't recent enough to include "Daesh".

(Aside: NLP programmers, from beginners to experts, don't make enough of a habit of using recent corpora and keeping them updated. The Brown corpus is 55 years old and is frequently used in tutorials to make models of English. The Penn Treebank is 24 years old. Pierre Vinken, the subject of the first sentence in the Penn Treebank, is dead.)

It's not a mistake. The algorithm puts words that mean similar things into the same bin.

So if say ISIS beheads people and SA beheads people then the algo will think they are the same, if people in SA throw gays off rooftops and ISIS does the same, then the algos think it's the same.

Algos are just statistical inference engines, they don't know you're not supposed to say certain things. It's kind of like when kids say something hilarious that no adult would say because the adult is supposed to know better.

SA is only pissed off because it's so close to the truth, like if bing translated Daesh to Norway no one would care because the comparison is ridiculous.