Hacker News Reader

Comment by turnsout | Hacker News Reader

Given how good some of the smaller code models are (such as Deepseek Coder at 6.7B), I'll be curious to see what this 70B model is capable of!

View on HN

AlphaCodium is the newest kid on the block that's SoTA pass@5 on coding tasks (authors claim at least 2x better than GPT4): https://github.com/Codium-ai/AlphaCodium

As for small models, Microsoft has been making noise with the unreleased WaveCoder-Ultra-6.7b (https://arxiv.org/abs/2312.14187).

AlphaCodium author says he should have used DSPy

https://twitter.com/talrid23/status/1751663363216580857

Is this better than GPT4's Grimoire?

Are weights available?

AlphaCodium is more of a prompt engineering / flow engineering strategy, so it can be used with existing models.

My personal experience is that Deepseek far exceeds code llama of the same size, but it was released quite a while ago.

Agreed—I hope Meta studied Deepseek's approach. The idea of a Deepseek Coder at 70B would be exciting.

There's a deepseek coder around 30-35b and it has almost identical performance to the 7b on benchmarks.

The “approach” is likely just training on more tokens.

It's the quality of data and the method of training, not just the amount of tokens (per their paper release a few days ago)

Phind [1] uses the larger 34B Model. Still, I'm also curious what they are gonna do with this one.

[1] https://news.ycombinator.com/item?id=38088538

Seems worse according to this benchmark:

https://huggingface.co/spaces/bigcode/bigcode-models-leaderb...

Reply on news.ycombinator.com