A better title would have been, "How Google fails at translations". The piece is actually interesting to read if you're into either language translations or how Google does stuff that seems magical, but the clickbait headline isn't doing it any favors.
"No, Google Translate was in no way useful to my translation of the 1956 Argentine novel Zama: let's get that out of the way first thing."
My guess is that that may be because she is fluent in Spanish and English and is a professional translator.
For the rest of us mortals who do occasionally attempt to read classic works in their original language, the answer to the headline is often "Yes, though not particularly well, but better than me".
So, the article notes that the Google translate of a challenging but not incomprehensible Spanish phrase hasn't improved in the four years the translator tested. I personally haven't noticed Facebook translate improve in any fashion over a similar period.
Yet I had heard of translation as one of the triumphs of recurrent neural networks. It may well be that such translations are doing better on some metric in some competitions yet that such formal improvements aren't palpable to humans yet. I have read previously things about translation now no longer needing language experts - which seems to say that perhaps automatic translation hasn't improved but merely gotten cheaper.
While this raises the "is this deep learning stuff hype?", I think AlphaGo shows that some deep learning stuff is pretty amazing. One might argue also that ConvNets have had the unambiguous triumphs which RNNs' triumph is a bit more ambiguous.
I'd be curious how folks who actually create this this technology would view progress of this sort.
I agree Google Translate hasn't improved in a while. It's probably because so much new research is focused on neural nets, not on improving the classic statistical translation approaches.
Facebook, though, I think they DO use NMT. Reasons:
1. They employ many famous neural net people, but not any experts on statistical machine translation that I'm aware of.
2. They came into the machine translation game fairly late. It would be odd to start off implementing an approach perceived to be on the way out.
3. Translation between closely related languages (e.g. Norwegian and Swedish) is far, far better on Facebook than in Google Translate. This is one thing I expect to be a strength of neural machine translation.
They can probably afford to burn more computational cycles, since it's not an on-demand service in the same way.
I just went back and checked again and the translation quality ranged from ungrammatical-but-broadly-comprehensible to entirely incoherent.
Of course, neither Spanish nor Tagalog is very close to English.
Still, this gives me the impression the RNNs represent at best an incremental improvement in automatic translation.
For example, it silently dropped the negation in a French -> English translation. Unfortunately, I can't remember the exact phrase but it was so semantically implausible--something like "regardless, we will be liable for any damages" that I went back and looked at the source text. It also sometimes replaces pronouns with oddly specific nouns: "He does X --> John does X" I assume these are both reflections of oddities in the training data.
http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...