At Mirage Studio we have successfully finetuned Llama 2 7B on a Dutch dataset to get it to output Dutch in a coherent way: https://huggingface.co/Mirage-Studio/llama-gaan-2-7b-chat-hf...
edit: To make the implied question explicit, I guess it might do well on other similar Germanic languages (say Norwegian) but struggle beyond that? Or?