Comment by joe_the_user | Hacker News Reader

Comment by joe_the_user | Hacker News Reader

Parent Full thread

Hmm,

So, the article notes that the Google translate of a challenging but not incomprehensible Spanish phrase hasn't improved in the four years the translator tested. I personally haven't noticed Facebook translate improve in any fashion over a similar period.

Yet I had heard of translation as one of the triumphs of recurrent neural networks. It may well be that such translations are doing better on some metric in some competitions yet that such formal improvements aren't palpable to humans yet. I have read previously things about translation now no longer needing language experts - which seems to say that perhaps automatic translation hasn't improved but merely gotten cheaper.

While this raises the "is this deep learning stuff hype?", I think AlphaGo shows that some deep learning stuff is pretty amazing. One might argue also that ConvNets have had the unambiguous triumphs which RNNs' triumph is a bit more ambiguous.

I'd be curious how folks who actually create this this technology would view progress of this sort.

Google Translate is not currently based on neural networks. The best NMT approaches are better than Google Translate, but they're currently too computationally intensive from what I've heard.

I agree Google Translate hasn't improved in a while. It's probably because so much new research is focused on neural nets, not on improving the classic statistical translation approaches.

Facebook, though, I think they DO use NMT. Reasons:

1. They employ many famous neural net people, but not any experts on statistical machine translation that I'm aware of.

2. They came into the machine translation game fairly late. It would be odd to start off implementing an approach perceived to be on the way out.

3. Translation between closely related languages (e.g. Norwegian and Swedish) is far, far better on Facebook than in Google Translate. This is one thing I expect to be a strength of neural machine translation.

They can probably afford to burn more computational cycles, since it's not an on-demand service in the same way.

I haven't found Facebook translate much to write home about either but I've mostly used Spanish to English and Tagalog to English.

I just went back and checked again and the translation quality ranged from ungrammatical-but-broadly-comprehensible to entirely incoherent.

Of course, neither Spanish nor Tagalog is very close to English.

Still, this gives me the impression the RNNs represent at best an incremental improvement in automatic translation.

I'd say that the fact that AlphaGo can play games against itself and there's a clear determiner what makes something successful(ie: a winning strategy), allows it to improve on it's own. I can't think of a way in which you could do this with language without having a human determine whether the output was readable/understandable or not.

This is pretty much it. The rules for Go are known. A perfect game of Go is an NP problem. We don't know how to frame the problem of Natural language interaction like that. We don't if it's even possible to do so.

One scary thing I've noticed is that Google Translate has a new failure mode. While it used to return obviously incomprehensible gibberish when things went off the rails, it now sometimes returns something plausible but subtly incorrect.

For example, it silently dropped the negation in a French -> English translation. Unfortunately, I can't remember the exact phrase but it was so semantically implausible--something like "regardless, we will be liable for any damages" that I went back and looked at the source text. It also sometimes replaces pronouns with oddly specific nouns: "He does X --> John does X" I assume these are both reflections of oddities in the training data.

Reminds me of the photocopier that started changing numbers in documents it copied.

http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

Wow. That must have been absolutely maddening to discover.

It's not a new failure mode, it has always done that.

I've wondered the same thing. I haven't noted any large improvements in Google Translate despite the RNNs demonstrating big gains for at least 3 years now. However, in all the public comments by Googlers I've read, I'm not sure they have rolled out RNN to Google Translate publically yet - this is the same company that created its own ASICs to save electricity on big deployments because Nvidia isn't moving fast enough.

Reply on news.ycombinator.com