latentsea

You know, I kinda relate to the feeling of not wanting look at those outputs if I think of it from a layman's perspective.

I just imagined that instead of math papers, they released 700+ feature length films, and the only way to tell if one of them is any good is to watch it in its entirety.

That feels pretty unappealing to me.

I know it's the same for human made films, so what's the difference right? But those are good enough most of the time that it's a decent bet, and the people that made them had real skin in the game.

Contrast that with something made by a nondeterministic slop machine with no skin in the game where small details can be off in a way that's jarring. Right out the gate I have an aversion to committing that much time to something that very well may waste it.

> And that is if you are lucky: if you have only IT skills, why should you be the first to get a fruit picking or plumbing job?

Easy. Because you have the skills to increase productivity by automating it. Oh wait...

One time this codebase I worked on had an ASCII art image of one of the devs in it buried in a comment at the bottom of some random file. YMMV.

None of it matters if in the limit the limitations of the model are reduced to zero.

> Or you want independence from the labs (fair enough)

This.

It's already unbelievably abstract as it is...

>There is a world where we get to the edge of AI capabilities, and we build on top of that.

We don't build on top of that. No need for us to. AI does. That's sort of the whole point of this endeavor is it not? Humans need not apply.

We should think about rejecting this AI future.

I use Qwen3.8-27B as a daily driver, and for things I know will be quite hard I tend to get ChatGPT to do the planning. Works very well.

anyone who works in software knows this term

Nope. You can run local LLMs on a 5060 Ti. No need for a 40k GPU.

Hard agree. This industry is awful now.

You really don't.

This level of hardware and cost just isn't necessary. You don't need to team up with your friends to go all in on a hardware purchase, which I've never heard of anyone doing anyway. Go spend some time over at r/LocalLLaMa, you'll see.

Qwen3.8-27B is all you need.

I guess future AI agents might market things as a "workman" for the same reason, despite less work being done by people :)

There has to be a software engineer named Claude whose mother or wife is named Karen. There just has to be.

Maybe use it and find out. Previously I was only able to run Qwen3.8-27B at acceptable (to me) speeds of 35 t/s on my R9700, but with Strata I'm doing 60 t/s running Qwen3.8-Flash-Next IQ3_XXS. I'm getting better results...

At the end of the day all that matters is if it works for you to complete your tasks. I'm getting better results faster with this now than I was with my previous setup. So... meh?

This wasn't a waste of time for me at all. On llama.cpp I could only run the IQ3_XXS quant at 21 t/s on my R9700 + system RAM, and on Strata I can run it at 60 t/s. Also... QFN at IQ3_XXS is giving better results for me than 27B at Q6 fwiw.

This is the conventional wisdom, but in practice what matters is how reliably the model performs on your tasks in the real world. I have an R9700 and an RTX 5060 Ti and I've been running an IQ3_S quant of 27B on the 5060 Ti vs a Q6 quant on the R9700. I still manage to get stuff done with the IQ3_S quant.

Good thing is you don't need your own technical expertise anymore.

Ah, I see you've found the AbstractExpertRemovalFactoryFactory!

You can run a 4 bit quant with this. Personally, I switched to running IQ3_XXS and am getting better outputs than 27B and at faster speeds.

You can run IQ3_XXS, IQ3_S, and IQ4_XS too. I've switched to IQ3_XXS and am running at 60 t/s on Strata vs the 21 t/s I was getting in llama.cpp. Better outputs too.

The benchmark indicates the IQ3_XXS quant beats 27B. I've switched to that now and am ditching 27B. Genuinely better results so far.

You can run IQ3_XXS, IQ3_S and the IQ4_XS quants on this too. It works. It's fantastic. I'm getting better results than 27B now.

We are already there. Prior to Strata the best I could run was Qwen3.8-27B at Q6, which itself is already at like Opus 4.5/4.6 level, and now with Strata on an R9700 and 64GB of RAM I can run Qwen3.8-Flash-Next IQ3_XXS at 60 t/s. It's even better. You can run it on even more modest hardware with Strata too.

Plus they announced Qwen4-Flash. It's not released yet, but it's the same architecture as Qwen3.8-Flash-Next, which now runs fast on consumer hardware.

Opus at home is a thing now.

> Qwen3.8 35ba3b

Does not exist. You're thinking of Qwen3.6 35ba3b

Well under the current US administration as non-US citizens we kinda feel this sentiment about the US now fwiw.