Yeah, this can be an issue. Today, there are few specialized LLM's of high quality so you end up having to use a massive all-in-one model like GPT-4 to reach for the language you need.

There's movement here though and it will get better. GPT-SW3 is a new model developed by AI Sweden, trained specifically on the Nordic languages only + English.

And beyond this, you have TrustLLM which is a new project that aims to be a large, open, European model trained on the Germanic languages to start with: https://liu.se/en/research/trustllm