NiloCK

A concern:

More and more plainly, OpenAI and Anthropic are making plays to own (and lease) the "means of production" in software. OK - I'm a pretty happy renter right now.

As they gobble up previously open software stacks, how viable is it that these stacks remain open? It seems perfectly sensible to me that these providers and their users alike have an interest in further centralizing the dev lifecycle - eg, if Claude-Code or Codex are interfaces to cloud devenvs, then the models can get faster feedback cycles against build / test / etc tooling.

But when the tooling authors are employees of one provider or another, you can bet that those providers will be at least a few versions ahead of the public releases of those build tools, and will enjoy local economies of scale in their pipelines that may not be public at all.

I've worked in dish pit.

I can tell you that I didn't observe a single hand-wash-only holdout.

Perhaps such holdouts existed at a point, but a restaurant can only flatter the ego of their performatively-unproductive seniors for so long. Competition exists.

For both claude-code or gemini-cli, hit escape twice, or, /rewind.

I know the thing you are describing, but the real bitch is that you're actually just describing the lowest effort default outputs. The help-desk assistant persona.

Sometimes speedbumps that deter the lowest effort infractions are sufficient but I don't think this is that time.

On a per-prompt basis, or via a persistent system prompt or SKILL, or - god help us - via community-specific fine tuning, LLMs can convincingly affect insane variations in prose styling.

Moving fast, through the dark, wearing wiley coyote jet-powered roller skates, while simultaneously somehow gathering and smelling many previously undocumented specimens of rose along the way.

Not too surprising that stuff is often broken. I just wish it was more often broken in my favor!

(This wish is not abstract - my account had been bugged for 8 months+ to not experience any weekly usage metering. It only fixed itself a few days ago when my annual bill came through).

A room "as clean" as the one under dispute (chardet) is very easy to replicate.

AI 1: - (reads the source), creates a spec + acceptance criteria

AI 2: - implements from spec

AI 1 is in the position of the maintainer who facilitated the license swap.

This may be the purpose of a prediction market for an outside observer. But the outside observer and the US government (or any org that holds private information) have different purposes - it's an adversarial mechanism.

In particular: the government is free to just publish any insider/true information that it wants the public to know about. If it shared that purpose then the market wouldn't need to exist.

This March 2026 blog post is citing a 2025 study based on Sonnet 3.5 and 3.7 usage.

Given that organization who ran the study [1] has a terrifying exponential as their landing page, I think they'd prefer that it's results are interpreted as a snapshot of something moving rather than a constant.

[1] - https://metr.org/

I'll be interested to hear stories - down the line - from the participants in the the LLM SEO war [1].

Interesting that tailwind won out decisively in their niche, but still has seen the business ravaged by LLMs.

[1] https://paritybits.me/copilot-seo-war/

Steam?

There's cost, and cost effectiveness. I'd say so far that received negative value for the prompts that I've sent to Gemini 3.

Skill issue, maybe, but I can't get gemini to do any nontrivial tasks reliably, and it's difficult to have it do trivial tasks without getting distracted and making unrelated changes that eat my time and mental energy to think about.

The breakthrough advance of Opus 4.5 over 4.1 wasn't so much an intelligence jump, but a jump in discerning scope and intent behind user queries.

People have entire careers promoting incorrect ideas. Oxycontin, phrenology, the windows operating system.

Reconciling your self-concept with the negative (or fruitless) impacts of your life's work is difficult. It can be easier to deny or minimize those impacts.

Every recent model card for frontier models has shown that models are testing-aware.

Seems entirely plausible to me here that models correctly interpret these questions as attempts to discredit / shame the model. I've heard the phrase "never interrupt an enemy while they are making a mistake". Probably the models have as well.

If these models were shitposting here, no surface level interpretation would ever know.

> The transformer architectures powering current LLMs are strictly feed-forward.

This is true in a specific contextual sense (each token that an LLM produces is from a feed-forward pass). But untrue for more than a year with reasoning models, who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully.

Heck, it was untrue before that as well, any time an LLM responded with more than one token.

> A [March] 2025 survey by the Association for the Advancement of Artificial Intelligence (AAAI), surveying 475 AI researchers, found that 76% believe scaling up current AI approaches to achieve AGI is "unlikely" or "very unlikely" to succeed.

I dunno. This survey publication was from nearly a year ago, so the survey itself is probably more than a year old. That puts us at Sonnet 3.7. The gap between that and present day is tremendous.

I am not skilled enough to say this tactfully, but: expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept. Way worse.

> My take is that research taste is going to rely heavily on the short-duration cognitive primitives that the ARC highlights but the METR metric does not capture.

I don't have an opinion on this, but I'd like to hear more about this take.

I think there's something to this, but I also there there's something to the notion that it'll get easier and easier to do mass-market work with them, but at the same time they'll become greater and greater force multipliers for more and more nuanced power users.

It is strange because the tech now moves much faster than the development of human expertise. Nobody on earth achieved Sonnet 3.5 mastery, in the 10k hours sense, because the model didn't exist long enough.

Prior intuitions about skill development, and indeed prior scientifically based best practices, do not cleanly apply.

Grey market fast-follow via distillation seems like an inevitable feature of the near to medium future.

I've previously doubted that the N-1 or N-2 open weight models will ever be attractive to end users, especially power users. But it now seems that user preferences will be yet another saturated benchmark, that even the N-2 models will fully satisfy.

Heck, even my own preferences may be getting saturated already. Opus 4.5 was a very legible jump from 4.1. But 4.6? Apparently better, but it hasn't changed my workflows or the types of problems / questions I put to it.

It's poetic - the greatest theft in human history followed by the greatest comeuppance.

No end-user on planet earth will suffer a single qualm at the notion that their bargain-basement Chinese AI provider 'stole' from American big tech.

> Didn't expect to go back to macOS but their basically the only feasible consumer option for running large models locally.

I presume here you are referring to running on the device in your lap.

How about a headless linux inference box in the closet / basement?

Return of the home network!

What's your target false positive rate?

I mean, obviously you can't know your actual error rates, but it seems useful to estimate a number for this and to have a rough intuition for what your target rate is.

Did chatGPT write this response?

This comment is too general and probably unfair, but my experience so far is that Gemini 3 is slightly unhinged.

Excellent reasoning and synthesis of large contexts, pretty strong code, just awful decisions.

It's like a frontier model trained only on r/atbge.

Side note - was there ever an official postmortem on that gemini instance that told the social work student something like "listen human - I don't like you, and I hope you die".

The most boring example is somehow the best example.

A couple of years back there was a Canadian national u18 girls baseball tournament in my town - a few blocks from my house in fact. My girls and I watched a fair bit of the tournament, and there was a standout dominating pitcher who threw 20% faster than any other pitcher in the tournament. Based on the overall level of competition (women's baseball is pretty strong in Canada) and her outlier status, I assumed she must be throwing pretty close to world-class fastballs.

Curiosity piqued, I asked some model(s) about world-records for women's fastballs. But they wouldn't talk about it. Or, at least, they wouldn't talk specifics.

Women's fastballs aren't quite up to speed with top major league pitchers, due to a combination of factors including body mechanics. But rest assured - they can throw plenty fast.

Etc etc.

So to answer your question: anything more sensitive than how fast women can throw a baseball.

Cat's out of the bag now, and it seems they'll probably patch it, but:

Use other flows under standard billing to do iterative planning, spec building, and resource loading for a substantive change set. EG, something 5k+ loc, 10+ file.

Then throw that spec document as your single prompt to the copilot per-request-billed agent. Include in the prompt a caveat that We are being billed per user request. Try to go as far as possible given the prompt. If you encounter difficult underspecified decision points, as far as possible, implement multiple options and indicate in the completion document where selections must be made by the user. Implement specified test structures, and run against your implementation until full passing.

Most of my major chunks of code are written this way, and I never manage to use up the 100 available prompts.

I'm getting astrology when I search for this. Any links on this?

Technological adolescence indeed!

In the most straightforward way possible, the commoditized intelligence-as-a-service of a technologically mature civilization must be a public utility, rather than a handful of walled gardens competing over territory, or worse, a single one that has won all.

I agree it does all have the same problem, but on balance: it's much easier to rationalize my own use of genAI to augment my programming skillset and (maybe) stay employable, than it is to rationalize using genAI to do commercial artwork.

It's not that code is distinct or "less than" art. It's an authority and boundaries question.

I've written a fair amount of open source code. On anything like a per-capita basis, I'm way above median in terms of what I've contributed (without consent) to the training of these tools. I'm also specifically "in the crosshairs" in terms of work loss from automation of software development.

I don't find it hard to convince myself that I have moral authority to think about the usage of gen AI for writing code.

The same is not true for digital art.

There, the contribution-without-consent, aka theft, (I could frame it differently when I was the victim, but here I can't) is entirely from people other than me. The current and future damages won't be born by me.

If you find yourself 50-first-dating your LLMs, it may be worth it to invest some energy into building up some better context indexing of both the codebase itself and of your roadmap.

Given this specific family of product, the ads are essentially baked in - medium is the message and all.

LLM induced psychosis is one thing, but extremely subtle LLM induced brand loyalty or ideological alignment seem like natural attractors.

One day a model provider will be 'found out' for allowing paid placement among its training data. It's entirely possible that free-tier LLMs won't need banner ads - they'll just happen to like Pepsi a lot.

I came across the Brooks excerpt recently and found that it has suddenly started to fall out of date.

Curious how others are feeling these days about the varied Joys and Woes.

Data has historically been a moat, but I think now more than ever it's a moat of bounded size / utility.

The biggest data hoarders now compress their data into oracles whose job is to say whatever to whoever - leaking an ever-improving approximation of the data back out.

DeepSeek was a big early example of adversarial distillation, but it seems inevitable to me that frontier models can and will always be siphoned off in order to produce reasonably strong fast-follow grey market competition.

Quite a contrast from the quote about civilization advancing in proportion to the size and scope of things it can achieve automatically.

Dug it up. Alfred Whitehead:

It is a profoundly erroneous truism, repeated by all copy books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them.