You can change the prompts if you want better results. But you don't have to do it. I've replaced GPT endpoint with LiteLlm a few times already and had to adjust the prompt only once. If I can't see the difference, why bother?
I think it would be really cool to get it running with something like Mistral in the browser: https://webllm.mlc.ai/