Hacker News Reader

I wonder if August of this year will be remembered like that too. The very first "opus like" local AI model came out this August (Qwen 3.8 Flash Next). I've been running it locally since for real programming and I consider it pretty much the same as opus 4.6 in coding ability (it lacks a bit in the factual knowledge area). It even exceeds opus on some tasks.

I've tried every hyped "open source" model before and all including latest models bigger than 1T parameters are pretty much toys.

This is the first one that isn't. It can't be overstated how huge of a deal that is. No more reliance on Anthropic.

Running this model to do real work is still not cheap. I run it on a pc with 6 rtx3090s and 192GB of RAM (and I use 90gb of that ram for KV cache). The model is entirely on gpus. It runs at 55tok/s 1500 prefill for one user at a time, and about 35tok/s 650 prefill per user for 5 simultaneous users. It doesn't seem like much until one realises you manage your own kv cache. You ca leave your sessions in cache for as long as you want. You can save them and restore 200k sessions a week later in a dozen seconds.

What many people don't realise is that usage of those models skews extremely heavily towards input processing. My claude code usage is about 1.3B tokens input per week and only about 8M output. On claude code I get 80% cache. At home it's more like 95%.

If this progress keeps up, and we get a fable quality model in a year in under 200B to run at home... Those "frontier labs" will be renting all of their gpus per hour not to go bankrupt.

What Quants do you run?

I am using qwen3.8-27B-UD-Q3_K_XL on a 5080 16gb card with 16gb of system ram at around 43-45tok/s with 65k context. I am using that model to set up containers on a proxmox server with two b70s and 96gb ram. The Q3 model implemented 10 different chat models, 2 image models, and 7 web based harnesses so I can compare. It can rewrite any part to improve it.

qwen3.8 is more than capable for software dev. The gemma models were terrible and fell apart during compaction. I think as long as the llm can test the results, you don't need anything being offered by a "frontier" model. The cloud AI is going to be used by people unable to run their own and they will eventually get squeezed on price.

The one tip I could give is compact before starting new steps or any action in the plan that is different than what was previously worked on. You want to manage what is in the context and don't want unnecessary work history details filling it up. You can always ask the llm to list the current plan, then compact after and do it more than once until you get the compaction <30%. This will be fixable by the harness that can choose better times to compact.

You don't want to start a new phase and have it compact a few minute after starting. This happening over and over again seems to potentially cause issues for long running sessions. Compaction slop that screws up what is in the context.

I can compare what I use at home vs paid models like astra at work and the difference is mostly meaningless.

What harness do you use?

"Those "frontier labs" will be renting all of their gpus per hour not to go bankrupt."

And those who go bankrupt, can sell their GPU's cheap, so I can indeed finally have my own fable.

I too use November 2025 as a true turning point.

Eternal November

The internet went from being dominated by academics, to normies, to bots.

It was. That's when I stopped coding most things by hand. The difference in what I could reliably get AI to do for me in February 2025 vs February 2026 is just massive. I immediately became a huge advocate amongst my peers for going all-in on automated engineering because this curve is about to get very steep and if you're not staying ahead, you might get left behind.

I'm genuinely curious: if the trend is toward "AI accomplishes my goals more easily than in the past", what curve are you "staying ahead" of?

Why isn't even easier for an AI noob to jump in at the next step with less friction than the current one?

the whole "get left behind" trope (as well as their entire comment tbh) is a classic ai psychosis thing

Well it's a good thing that isn't what I said or implied, and that you instead smugly misinterpreted my comment.

The person you're replying to had the decency to ask me for clarification; for you, I'd suggest brushing up on your reading comprehension and learning to respect the rules of this site, which include interpreting comments in their best light and focusing on positive, substantial contributions instead of negative, inflammatory posts.

maybe don't say things like "if you're not staying ahead, you might get left behind" since, like I said, it's a trope at this point.

Again, I'm going to refer to the advice I just laid out for you. I'm not responsible for you and I don't need to cater my words to your lack of reading comprehension.

I think this is both true and untrue. For a long time, I was an AI skeptic, perhaps even a hater. A chauvinist for writing code by 'hand.' But it's become very difficult, as I've gradually migrated to AI-authoring of code, to go back to writing code by hand and retain the same velocity.

Part of this is certainly that my hand-authoring code skills have atrophied, sure, but my workflow has also radically changed. Previously, I would spent a lot of time and focus on a single work item, and only context switch to other tasks whenever I would wait on CI or a long build. It meant that I spent a lot of time understanding one thing at a time, and interruptions (forced context switches) incurred a massive switching cost.

Now, having moved to largely AI-authored code, I find myself necessarily working on multiple threads at the same time. This means I can meaningfully progress each of those threads in parallel, with a much-reduced overhead on context switching, since I don't have my head down focusing on all the details of the work. And if the task really demands it, I can still stop and focus on one thread to sketch out the code manually, think about the concepts more deeply, etc.

It's a very different workflow, and there are certainly downsides, but the upside is that the rate of which I've been able to put up good-quality PRs has measurably increased. It's not quite 2x, and it's certainly not 10x, but it's definitely noticeable. I do understand a bit less, but there was always more work than time to understand things in full detail. I guess only time will tell if that missing understanding was actually vital to the long-term success of my work.

yes that's fine, i have a similar experience. but there is a difference between saying what you did, and the type of AI religious fundamentalism where you got "converted" when truely good AI coding models revealed themselves to you, and then you go around trying to save those who would otherwise be "left behind".

AI is becoming a more and more powerful lever that lets you move greater loads per application; it is in a real sense giving you more leverage. But the lever is difficult to use effectively, and the methods for using the level change every week or so.

> methods for using the level change every week or so

hmm doesn't mean noob will become better? if what I'm "learning" about AI is going to get deprecated every week... then it doesn't make sense to "keep up"

Here's how I think about it. There's two different things "knowing how to use AI effectively" and "knowing how to leverage effective AI to improve your project".

1. Both are multipliers on skills you already have. A "noob" will be at a disadvantage.

2. The second one is a complex blend of technical skill, domain knowledge and human factors (knowing your user-base, knowing your product/project, understanding UX/DX/whatever) that you probably will always have an edge on.

It requires skill to prompt the AI to do best possible work. I have senior colleagues that produce horrible slop and others that always produce great results. Managing the context, knowing when to stop the AI. Knowing what to question, what to "trust" in is a big deal.

Also knowing what models are good for what. There was time half a year ago when Google genuinely had a better model than everyone else. I used it for everything and 3 weeks later they lobotomise it (sorry "optimised") and I went back to opus...

I think Anthropic is like a drug dealer, giving us the sweet sweet drug for free (I don't think our $200 a month subscriptions even cover the electricity for our use) and the time to pay will come very soon...

I expect this subscription will cost $2k a month. Will it be a normal increase? Or will the "enshittify" existing models to the point you'll pay $2k to get fable 6 to do what opus 4.8 did fine in July 2026?

I was referring to the amount of people in the future who might be employed at today's engineering wages, and the differential between those people and the average employed engineer today.

What will that differential be? A large component will be social reinforcement: generational wealth, connectedness, and such. Things like merit might take more of a backseat. So people who do not have the necessary social capital may be fighting each other for very limited amounts of positions.

That is the steepening of the curve for people like me: I was homeless at 16 and finished high school on my own, was given a full ride to LSU, grants, room and board and a job in the Comp Sci department, but lost all of it after an immature and vindictive high school teacher illegally modified my grade in a core class in order to fuck me over. I didn't have parents to back me up at the school board and make things right.

Instead I suffered through years of homelessness and had to find my own path into the industry by starting companies with friends and doing all of the engineering. Since then I've led multiple teams, made some connections, shipped a lot of cool stuff and bring to the table a wealth of experience and a generalist skillset that is both wide and deep. Yet, I too wonder where my place in this changing industry will be once things settle a bit. Probably less engineering and more focus on business development.

The flipside though is as you've said: The fruits of engineering are more accessible than ever to the layman, and individuals can currently possess an unprecedented amount of agency and leverage. I think that is amazing and am fully behind it. I do know that it means the process of renormalization is going to be very rough, given the similarly unprecedented rate of industry change these technologies are bringing.

There are two types of AI users: those who chase the rabbit and those who do not. Some people bounce from tool to tool QuantumLeap-style hoping that maybe the next tool will be the big thing to solve all their issues. Other people use AI to do actual work. Once they find a tool that works, they stick with that tool until they feal a need to upgrade.

It is like people fishing. Some people go out and catch fish with the tools they know will work. For other people, every day at the lake requires a new boat/rod/lure. They spend more time figuring out how to use their new toy than they do catching fish.

Because improved models seem to increase, not decrease, the gap between what you get from really expert supervision (prompting, steering, hand-corrections along the way, etc.) vs what you get from fairy-dust and wishes.

AI will eventually get better than you at all you do, because it trains on your inputs.

Secret knowledge/data will be tomorrow's gold.

> if you're not staying ahead, you might get left behind

that sounds ominous. what do you mean?

Yeha me too but I need to remember Opus 5.5 aka September/October because i wouldn't expected a model feel again relevant different but it does.

Seems to me we need tooling to do automatic offensive security as soon as a new frontier model comes out, with quickly turned around patches using the same frontier model. Rinse and repeat. Virtuous agentic security loop.

As soon as a new model drops I point it at one codebase I have and ask it to do a security audit, look for bugs, check for optimizations, things that are over-engineered, etc.

Every single time I've done this it has found at least one serious bug or security hole.

Makes me wonder how many are left I've not found.

Isn't that part of the cloud AI business model now? Businesses have to pay for early access so they can weed out any new bugs before the model goes public. Which is kind of crazy, because you are paying for early access to protect yourself from other paying customers of the same cloud AI model.

Fun part is July 2026 „summer of bliss” for cURL project.

As much as cybersec forums were outraged that everyone will be hacked because of that — nothing happened.

Do you think that eventually one would 'fix all holes' or is this an eternal fight against the windmills?

>> We fight AI fire with fire, and hand-review, of course.

Wouldn't it be nice if AI vulnerability reports came with AI pull requests to fix them? The thinking context that found it should be readily able to propose a fix. It would still need review but even when AI PRs aren't right they often point in the right direction.

Why? Imagine that you are a competing, close source product/project. Just bombard your competition with AI reports, and let them drown in misery. Problem solved! (/s)

This just goes to show that yes, if security researchers were to do that, it would great, but they are not the only actors here...

I'll speak up in defense of the reporters: FWIW, so far my strong impression is we're hearing from independent security researchers. Right now, for us, so far (enough qualifiers yet?) the system is working for us: independent security researchers are farming reputation by finding real problems. That's not a bad thing.

And in most cases they do propose solutions, although we generally resolve the issues on our own.

There's a small percentage where we make the case that the ticket is not a real vulnerability, and then we have to grit our teeth through repeated reports of the same "vulnerability." But it's a small percentage so far.

We do typically have to reconsider the severity. The researchers understandably want to see everything as a nine...

That doesn't really help it's just more stuff to review and question if it makes any sense at all

Yeah just like with Navier Stokes.

If it's a PR created by just prompting with the vuln report, I might as well do it myself.

wonder how much this can easily speed up the malicious-contributer attack. where someone suggest a security fix in a extremely obscure and irrelevant code, but the fix actually adds a new condition that then can be exploited elsewhere.

Absolutely possible, which is why I refuse to take my eyes off code review (mine, or a few other trusted souls)

For me the November Singularity was when ChatGPT was released in November 2022. Of course, it was a faaaaaar cry from what we see today, and it took a lot of careful wrangling, but it could write reams of correct code and tests even back then.

The thing was the wrangling was relatively straightforward, if cumbersome. Largely, it involved being very precise with the context and instructions it was given. I could imagine a lot of that getting automated (i.e. what we today call harnesses) or recursively addressed by creative meta-prompting. Supported by similarly conceptually simple advances like chain-of-thought reasoning I suspect that is the biggest thing that the models have figured out what to do today compared to then: manage themselves carefully.

Although I could not have predicted these exact outcomes, the implications for everything that is unfolding now were clear even then.

What's this got to do with the "several vulnerabilities discovered in the Linux Kernel"?

If this year has taught me anything, it's that there was never anything like stability in software and that stability is the wrong optimization goal.

Instead we should aim for quick updates, strong isolation and sandboxing.

Whatever that means for TDD and other methodologies that seemingly all have failed to encode guardrails in the development workflow.

I hope that's also a reference to the November Revolution :D

I like the November Singularity, I hope that catches on. That was definitely the point where I went from "AI is overhyped" to "oh shit the hype bros may be on to something"

Yes, good name and I also feel that was a major inflection point. Up until then I found all AI models to be terrible at programming, with the difference between GPT o3, Sonnet 4 and every other model since GPT-3 being just the exact flavor of terrible they were.

November 2025, with Opus 4.5, was the first time I was impressed by an LLM doing something non-trivial with a reasonably good level of quality.

Yes, and Qwen 3.8 27b is a similar inflection point for local, although I wouldn't claim it's quite as good as Opus 4.5 it is genuinely useful. Whether that actually matters will depend on whether it ever becomes the most cost-effective tool for the job, but it's impressive as all heck

Yeah before Qwen 3.8 27b all local LLMs did feel like not very smart chat completion models. You would hit the limits of their intelligence all the time. Chatting with Qwen 3.8 is night and day, and it can do some pretty damn good coding and agentic work for its size. IMO its better than where frontier labs were at about 18 months ago, which is mind-blowing for a small locally hosted model.

Looks like someone coined it almost immediately! https://agi.co.uk/november-singularity-ai-agentic-era/