Israeli Translator Destroys ChatGPT Hebrew Translation of the Great Gatsby

Whether you find ChatGPT useful depends a lot on how risky it is. Something like the static labels on an app, or translating a brochure ecommerce site is low risk: you don't mind if a few Greek/Norwegian/etc people think your page is slightly wrong, you might still convert them.

High risk would be regulatory things like legal documents, operation manuals for certain things, that kind of thing.

I think we'll end up with a Mini-mills story here. All the crappy little translation things nobody cares about will go into an LLM, with time it will improve, and then it will flip the industry on its head.

I don't entirely disagree, though what I got from the video is that it's generating linguistic inconsistencies, i. e. literal translation stitched together and sentences that don't make sense, which is surprising for a language model. The addition of words that aren't in the English I can understand. Otoh, does chatgpt represent itself as know Hebrew?

Very good point. I feel this is a moment for everything like AlphaGo was to game Go, we’re at the moment right before beating the grandmasters. Humans still have some glimpses of being better computers to some very specific tasks but not for long.

Just showed the video to my wife who is also a translator (though usually FROM Hebrew TO English rather than the other way around) and she also was not very impressed with how robotic the chatGPT translation was. This is the kind of translation quality you could already get out of Google Translate and the like.

With regards to the chess-playing dog analogy: it is anecdotal evidence but certainly in the company I've been contracting for they've been very excited about translating the website in 24 different languages with chatGPT alone. This is based on the CTO playing around with chatGPT translations for 30 minutes and thinking they were "quite OK". IMO this would be like entering the dog into a serious chess competition and then taking out a loan to bet on the dog winning any prizes. Machine translation is not on a professional level yet, even with chatGPT.

TBH it is difficult to see how the current LLM based approach would succeed in proper translation work, since almost by definition things like puns and clever wordplay are not going to be the most likely tokens for a model to predict.

The "Artificial General Intelligence" fails because it wasn't trained to do this?

> an LLM that hasn't been trained to do translations can do some pretty impressive translations

An LLM that hasn't been trained to do translations can do translations, badly. I mean, if you want bad translations, you don't need to whip out an LLM, there are lots of simpler methods that will also give bad translations.