Hacker News Reader

Comment by XCabbage | Hacker News Reader

Obvious thought that I haven't tested: can you literally achieve this by getting it to answer in Dutch, or training an AI on Dutch text? Plausibly* Dutch-language training data will reflect this cultural difference by virtue of being written primarily by Dutch people.

* (though not necessarily, since the Internet is its own country with its own culture, and much training data comes from the Internet)

View on HN

That hardly works. Though from my limited experiments, claude's models are better at this than OpenAIs. OpenAI will, quite often, come with suggestions that are literal translations of "anglicist" phrases.

Such as "Ik hoop dat deze email u gezond vindt" (I hope this email finds you well), which is so wrong that not even "simple" translation tools would suggest this.

Seeing that OpenAIs models can (could? This is from a large test we did months ago) not even use proper localized phrases but uses American ones, I highly doubt it can or will respond by refusing answers when it has none based on the training data.

See https://chatgpt.com/share/67cfaa34-78fc-8009-816f-ca5778c93e... and https://claude.ai/share/d03ce2bf-0bbd-4b91-9afd-eabfc713e99a

With ChatGPT O1: https://chatgpt.com/share/67cfaa7e-70ac-8009-871b-571924b5a5... and with Claude's 'extended': https://claude.ai/share/4ba55410-98f3-4b53-9540-219acd2cdc4c

Some of the examples are still wrong. Nuanced, but a Dutch native will still frown at it.

But more importantly is that you limited the context a lot. As in: the scope, the prompt, is very narrow.

In our case, we were generating emails. Lines like greetings are but one of 20+ details in that mail and not even the most important ones. The prompts ever larger, the multishot examples ever more tuned. And then, one in a few hundred will turn up with these "horrible" translations.

We've now moved to a chain of models, where we generate emails in American (the creative part) and then use another model to translate them to Dutch (the non-creative but culturally aware part). This works much better as we can pick models that are good at one thing or tuned to do this one thing better (either by the LLMAAS provider, or by parameters such as temperature).

> Some of the examples are still wrong. Nuanced, but a Dutch native will still frown at it.

Thanks, that why I posted the links here: I couldn't judge by myself how good these creations were.

> But more importantly is that you limited the context a lot. As in: the scope, the prompt, is very narrow.

Yes, I just did a very crude experiment.

I've tried Dutch answers and it is more than happy to hallucinate and give me answers that are very "American". Doesn't help that our culture is very inspired by the US pop culture as well since the internet.

Haven't tried prompt engineering with the Dutch stereotype, though.

I wonder (if this works at all) if the effect might be stronger if you also prompted in Dutch, preferably written by a fluent speaker rather than machine-translated.

Internal vectors aren't in any particular language.

No, but theoretically, blunt responses might be more common in Dutch-language training data. A well-fit model would be expected to replicate that. (Basically similar to straight up asking it to be more blunt, except it has probably trained a lot more on _Dutch_ than on _someone just told me to be blunt_ so the effect might be more natural and more subtle.)

Reply on news.ycombinator.com