It’s not the same is what I’m saying. Google Translate was never good. GPT is literally perfect translation.
It literally is not. GPT as MT to users is just a near-SOTA alternate implementation of machine translation, though things like open-weight 13B models at edge has huge potentials so "just" might not be the right word in that sense. I've tried round tripping a line from your comment with ChatGPT 3.5 and Google Translate - I'll be honest, I did try couple different snippets to find one that breaks worse in GPT, but GT is sometimes just more accurate:
OG: Sometimes I have it explain something in English for me and Japanese for him and vice versa. It can switch up the languages in the same context no problem. All kinds of phrases in both English and Japanese that are hard to explain to each other ChatGPT has no problem helping us out.
GPT: 時々、私は彼には日本語で説明し、私には英語で説明してもらうことがあります。言語を同じ文脈で切り替えることもできます。ChatGPTは、英語と日本語の両方で説明が難しい様々なフレーズについても問題ありません。
GT: 私には英語で、彼には日本語で何かを説明してもらうこともあれば、その逆も同様です。 同じコンテキスト内で言語を問題なく切り替えることができます。 お互いに説明するのが難しい英語と日本語のあらゆる種類のフレーズを、ChatGPT が問題なく助けてくれます。
GPT->GPT: Sometimes, I explain things to him in Japanese, and he explains them to me in English. We can also switch languages within the same context. ChatGPT has no problem with various phrases that are difficult to explain in both English and Japanese.
GT->GT: Sometimes I'll have him explain something in English and he in Japanese, and vice versa. You can safely switch between languages within the same context. ChatGPT will help you with all kinds of phrases in English and Japanese that are difficult to explain to each other.
Second I’m giving you feedback from an actual Japanese person who used Google Translate all the time to go back and forth. Now using GPT 4 they say it is no contest way better than Google Translate. To a point where they can use it to understand and communicate professionally with English and Japanese business customers.
But you can even see it yourself taking actual articles from Japanese websites and running them through Google Translate versus GPT 4, GPT translates into much much more fluent English, same with the translation to Japanese.
If you could read all of these outputs and compare against each others(thanks glandium-san and sorry for bothering), you'll just see all of it has pros and cons and that they compare. They're not worlds apart, in fact I think GT did best in this instance probably from massive accumulation of edge case fixes or some other non-scaling improvements behind it.
Like I said, it's an interesting phenomenon - it could be UX advantages, or friendlier tones, latency, personally I think it's dataset imbalance or could be numerous other things. But when it comes to actual literal translation performance, um, it's just an MT.
GPT4->GPT4: Sometimes, I have explanations given in English for me, or in Japanese for him, and vice versa. Switching languages in the same context is no problem at all. ChatGPT also supports me with various phrases that are difficult to explain in both English and Japanese.
DeepL: 時々、私は英語で、彼は日本語で何かを説明する。同じ文脈で言語を切り替えても問題ありません。英語でも日本語でも説明しにくいフレーズもChatGPTは問題なく教えてくれます。
DeepL->DeepL: Sometimes I explain something in English and he explains something in Japanese. Switching languages in the same context is no problem. ChatGPT has no problem teaching me phrases that are difficult to explain in either English or Japanese.
The main problem is the implicit subject due to omission of pronoms in regular japanese, among other ambiguous things that can only be correctly translated in context. Also ommision of gender, plural, etc. The huge decoder-only models so far do a much better job grasping/guessing that and staying consistent compared to small encoder-decoder MTL models.
This simple phrase any beginner in japanese would understand: "山田さん、猫いますか？はい、います。" "Yamada-san, neko imasuka? Hai, imasu."
Google Translate now: Yamada-san, do you have a cat? Yes, I'm here.
When tested a few years ago: “Mr Yamada, does he have a cat? Yes, it has a cat”
LLama-3-8b-instruct: "Yamada-san, is there a cat? Ah, yes, there is."
GPT 3.5: "Does Yamada-san have a cat? Yes, he/she does."
gemini-1.5-pro-api-0409-preview: Does Mr./Ms. Yamada have a cat? Yes, they do.
LLama-3-70B-instruct: "Yamada-san, do you have a cat? Yes, I do."
GPT 4, Command-R+: "Mr. Yamada, do you have a cat? Yes, I do."
All LLMs gave possible translations, but the larger the LLM, more likely to be correct the translation became. Well, the top two ones chose a gender, while the -san thing is a clever way to avoid chosing a gender, but not a very pure translation. If I had to give only one translation, I would pick either of those.
Also of note, while some even explained each element of their translation, none said that other possible translations were possible.