German AI consortium releases Soofi S, an open 30B model that tops benchmarks | Hacker News Reader

German AI consortium releases Soofi S, an open 30B model that tops benchmarks

the-decoder.com

147 pointsamai34 comments

Open articleView on HN

It tops benchmarks because it uses them in its training data. https://x.com/eliebakouch/status/2077425801633427919

https://nitter.net/eliebakouch/status/2077425801633427919

That is very disappointing.

That's just embarrassing. It keeps happening though.

Disappointing. Sounds like Volkswagen 2.0

training on the test set is all you need

It's ready for law school!

scrape the scrapers... all's fair...

looks like they have since fixed the dataset, hope they release an updated report (either without the leaked evals or a new checkpoint/weights)

https://huggingface.co/datasets/AIML-TUDA/QA-base/commit/2f9...

Reminds me of: https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal

It seems they corrected this already: https://nitter.net/jphme/status/2077468512025477226#m

I guess you didn't read ellie's reply directly underneath that?

> the facility runs entirely on renewable energy, is cooled with water from the Eisbach canal, and feeds waste heat into the surrounding Tucherpark neighborhood.

That's it right there. Show the others how it's done. Can see going forward most model training being done in winter, making for a proper resource recycling ecosystem.

It's funny, my reaction was the exact opposite. Details like this show that they're fundamentally unserious and focused on the wrong things. Imagine if Germany, when developing their automotive industry, spent all their time focusing on reusing the waste heat from production to heat homes instead of just building great cars. They probably would not have sold many cars!

Imagine Canada being on fire due to global warming while you write this comment.

Survival! Wrong thing! Stay away! Sell cars, that's what you've got to do, at any price.

But then again, AI is a paperclip maximiser after all...

Oh no!

Must resist... making... papercli

That is funny because Germany did in fact use the process ("waste") heat from steel mills for district heating on a large scale. The steel was then used by the automotive industry, among other things. We're talking 50+ years ago.

In most German cities (and even many villages), district heating is a rather common utility offering. There's pipes everywhere to move around the hot water to people's homes, and the water is typically heated by the waste heat from gas plants and waste heat from factories (though increasingly being supplemented with giant heat pumps).

This isnt some novel thing in Germany, selling excess heat from some energy intensive commercial proccess is pretty standard. The local district heating operator probably took care of almost all the infrastructure for them.

Two weeks ago a heatwave linked to climate change killed in Germany an excess 1300 people [1, 2]. It was the deadliest single day of the last 25 years [3] - the mean numbers of deaths per day is 2400 while June 29 reports 4800.

If the attitude to a German AI effort using renewables is to object "but think of the benchmarks!" that feels like a step in the wrong direction in general and, with this particular context, also in particular.

[1] https://www.bbc.com/news/articles/cn4d2vv935lo

[2] https://www.dw.com/en/germany-june-heat-wave-linked-to-5000-...

[3] https://datawrapper.dwcdn.net/gvL9c/4/

District heating is the most normal thing ever....

you realize chinese models are beating american models because they have limits to their compute.

structural limits improve inovation, not the opposite.

which chinese models are beating American models

Your attitude suggests you are unserious about the future of the human race.

So instead of making efficient use of resources you say it is better to waste them ? Even if you want to take this from an economic PoV - people are paying for that otherwise waste heat you are supplying them.

Great to see more competition in this space, and especially from Europe, but... it's a shame when the benchmarks don't include the current best comparable models. They shows results against Qwen 3.5 and Gemma 3, but Qwen 3.6 and Gemma 4 have been available for ~three months.

I bet it's doing really well against Llama 3 :)

I think this news is more about the infrastructure than the model.

Either way, happy to see two open models from outside the us-cn duopoly in the same week.

Some other (as in better) sources I found:

- https://huggingface.co/spaces/Soofi-Project/Pretraining-Tech...

- https://arxiv.org/pdf/2607.09424

How can we get these things to fight each other for sport? Real battle bots

"Fully open source model" with "Long-term, license-free availability for industry"

Nice trick to be not comparable with most other LLMs on the market, open weight or proprietary.

But I think, that's the right way.

I wanted to try it today.

Had to request access on huggingface, which was still not answered after a day.

Given all the Inkling and Kimi announcements, this atm just feels like too little too late to be taken seriously.

That said, it’s their first release and I hope they do catch up fast.

Better late than never. Germans have entered the chat

More like Nemotron 3 with benchmaxxing enabled has (re-)entered the chat.

Reply on news.ycombinator.com