Apertus is the open source 8b and 70b LLM from swiss-ai. They've published both the base and the instruct sft models. Very cool that projects like this exist.
Focus was on open data, languages and auditability.
Their loss function is fancy, not sure about the effects