maxignol

Hi ! It's been a few months since with my best friend (and cofounder) we were really getting sick of one rate limiting while the other had plenty of "tokens" left on his Claude subscription. So we made this. And we find it cool so we share it to you.

Please give us feedback, that'd be really appreciated, negative or positive ;)

Author here. Two things that are worth repeating up front: corpus A is 84% of the pooled bill, so the pooled −0.01% is pretty much "corpus A plus noise" so read it per-rows instead. And p_fire is a lower bound because of the detector's sensitivity taken as 1 (which it obviously isn't). Caveman's real fire rate is somewhat higher than 50.6%. Code and commands: github.com/jaynapp/jayn-caveman

> I don't know if engaging with that button means I'm less likely to be served such content, or whether it's cleaning up everyone's feeds.

I would guess it's both cause I've been having less AI slop on LinkedIn for the past few weeks (yet not none).

Been laughing for the past 10 minutes, thanks for this one. (I tried the death of my cat by car in front of my house in a Go Girl manner, absolute cinema)

Optimizing speed is really the way to go. Yet 24GB is not what everyone can afford. Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.

Is the model picked through the router only for the first user turn or is there multi-turn routing (or planned to be added) ?

I'm really excited about what's been happening couple last weeks for local inference. I feel like it all started after colibri [1] was released. Great work !

Anyone got recommendation about what local model to use for what purpose ? I feel like (as they were saying in moonshot blog post [2]) each llm can be an expert in its own categories and with several small local we might get good coverage for decent usage, granted each one is specialized enough.

[1] : https://github.com/JustVugg/colibri [2] : https://fireworks.ai/blog/kimik3-fable

I dont quite understand why GGUF is better optimized. Are the performances better for the same amount of VRAM ?

> I’d suspect the harness to massively affect token use and optimisation

Yes they do according to databricks -> https://www.databricks.com/blog/benchmarking-coding-agents-d...

That's why I find comparing models on benchmarks only gives the tendency. We should be comparing model x harness to have accurate metrics.

What is your harness with every model ?

> my use cases stop aligning to swebench pro around 50% accuracy, and more closely align with DeepSWE.

What do you mean by that ? If the model is higher than 50% on swebench pro then it tends to drift from what you like it to do, like DeepSWE benchmarks ?

Shouldn’t we fear they start doing only close source like most us labs once they catch up in market shares ?

Well, I had not heard of RLM before, just read the paper, thank you for introducing me to your lazy version !

I’ve never seen the Id approach before, that’s a good idea ! Though I was wondering how do you manage to keep costs low within the 7 agents ?

Giving the accuracy-token graph and not the accuracy-cost graph, thus we cannot easily compare costs with other models, is not a way to gain my trust

I’m truly impressed by your work ! I don’t know if this is planned for near future, but how about adding energy efficiency benchmarks ? Because running locally is a great feeling, but the electricity bill should not be forgotten

Well, had a meeting with a VC that literally brought his note taking 3rd person to the meeting, did not even say hello. Tbh, I still find it creepier than an AI transcriber.

This seems really bad…

I guess the technology used here must be ground-breaking lol

Did not seem to find how much tokens per second he achieved with this setup ?

Thus the prices in switzerland are higher than anywhere else. I don’t know about you, but I’d have no use of 25gbits/s

Lol next time I’ll just apply with 4 accounts and maybe get in once.

Are many people using HackerRank ATS ?

Have you tried opencode go ?

Would you recommand some ressources about how multiple neural engines are used in data centers ?

I guess it was inevitable. Is it only RAM related ?

Funny one x) Though I ain’t sure if even more data is useful on hackernews

Great way to honor Tony and his work

In the end, sorting prs and vulnerabilities has been the same for open source maintainers. How about adding a credibility score to every github account ? Couldn’t that cut sorting times ?

3B param on par with opus 4.5 sounds interesting. Will read the full article before making my mind