Winfred-zz

I just ran a set of benchmarks, ninfer-3090-qwen3.8-27b (so mix of Q4 and Q5) vs strata-qwen3.8-flash-next-iq3_xxs (so Q3):

│------------------- │ Ninfer-3090 │ Strata

│ Code generation │ 52/78 (66.7%) │ 70/78 (89.7%)

│ Code completion │ 40/50 (80.0%) │ 44/50 (88.0%)

│ Total------------- │ 92/128 (71.9%) │ 114/128 (89.1%)

│ API failures------ │ 10 │ 5

- Ninfer generation: ~122 min total.

- Strata generation: ~142 min total.

So strata is a little slower, but keep in mind that ninfer-3090 is very optimized for a Qwen 3.8. Standard Qwen 3.8 runs at 20 t/s, this modified version can do 50 t/s (but it's extremely long in it's thinking, it just goes on and on.

This is on a 3090 that will crash unless power capped, with a Zen 2 CPU, 64GB DDR4 with a PCIe that refuses to go higher than 8x (basically pretty crappy all in all).

Yet with some tweaking and optimizing I still manage to get strata to run at 40 to 60 t/s.

That strata has been optimized on my Oh My Pi conversations. So when I'm using it, it's probably faster and closer to ninfer in speed than during those unoptimized benchmark tests.

I mostly use Oh My Pi.

Oh My Pi comes preloaded with most stuff you need, so there's not much need to add extensions. https://github.com/can1357/oh-my-pi

In my own experience, qwen3.8-27b 4bit can consistently find bugs in software written by sonnet 5 and opus 5. But it does do that at maybe 1/10th the speed. Still a pretty good deal if you're coding without wanting to spend big.

qwen3.8-27b 4bit has a following specifically for being exceptionally gifted for such a small model.

It's containerized. It's an easy enough problem to solve, if you don't want it complicated.

Here's what I did with my own setup: Stop the containers at night, copy the files to a second system that's configured with the same containers and that can become the primary as needed.

I already monitor my systems for health and backup states, so these would be added to that. But all that does take a few hours to set up.

You want your backups regardless and you want to have some certainty they're working/good. It's a problem that needs to be addressed, even you are going to agree with me on that.

Of course I do. There's also chairs in case there's an issue with the couch.

I have so many projects on my computer where it doesn't matter if I know how it works or not. Probably well over 100. I think closer to 200, depending on how you count.

They're relatively simple, they do the task they need to and then they wait until they're needed again (or not).

In the past I wrote them, then forgot how they worked, until I needed them again, relearned what I did and adapted it.

Now I just don't have to know how exactly they work, I just get an AI to read the documentation anytime I need to reuse the project and I'll query the AI to fill in the details and to make changes and I ask the AI to run the code and debug it.

Perfect use cases for today's AIs. Doesn't even require SOTA, I can run comfortably on a Sonnet 5 or a Qwen 3.8 and it'll do exactly what it needs to do without making too many mistakes.

Not all code is large corporate code bases.

>At the core, it's just an Ubuntu machine with a web server that hosts a dashboard and routes HTTP(s) requests to containerized apps.

So no fail over/redundancy? I think at minimum it should be two machines, so one can break. That is one of the important features of cloud applications as far as I'm concerned.

You don't have to deal with the problem of a single computer breaking.