That said ChatGPT is better at picking up dialect nuances. I can ask it to tell me how to say something in Egyptian Arabic, vs most translation tools will use Modern Standard.
It’s not very interesting at all to hear about ChatGPT-3.5 making mistakes, and an unlabeled reference to “ChatGPT” usually seems to mean ChatGPT-3.5. ChatGPT-4 is an entirely different level of product.
ChatGPT-4 can make mistakes too, and those cases are much more interesting since it is among the best LLMs on the market, so it is more representative of what current LLMs can (and can’t) do.
For reference, here is ChatGPT-4 discussing this: https://chat.openai.com/share/f67600f5-7d36-49b7-b136-f77dcb...
I don’t know that language, so I can’t say how accurate it is. For the languages that are represented well in its training data, ChatGPT-4 seems capable of really good translations.
I've never heard of titli makhi before, but have heard of bher from multiple sources. I don't know definitely (as in I haven't looked it up in a formal dictionary) but colloquially in my dialect bher is more correct.
Also worth noting that GPT-4 used the incorrect letter representing the R. Google translate correctly provides: بهڑ
Thanks for the added context though, it's super interesting.
I just tried again and it said "shahbaz" which is a name or at best an adjective. The correct translation would be "bher" which Google Translate displays as the "alternative" translation.