ekidd

Still, this is not a bad situation overall. If you are forced to trust the C compiler to correctly compile C, and the hardware to correctly implement the instructions in the documentation, well, you're already forced to trust both of those every day.

So this doesn't give you an absolute proof of correctness. But it does substantially reduce the size of the problem, which is now limited to (1) verifying that you actually proved what you think you did, and (2) all the stuff you were normally trusting anyway. (Some of the stuff you were trust anyway is broken, of course.) But this is a smaller problem than trusting 1,000 lines of highly-optimized CSG code written by a model we don't actually understand.

> My experience so far, has been that it's barely a junior dev.

At least on greenfield projects, Fable is more like having an endless succession of fly-by senior devs who lock themselves in an office for a week, and who come back with very reasonable code that I then need to maintain somehow.

For longer-term maintenance, I am actually slowly warming to Sonnet 4.5-era models (so October 2025, right before the Opus revolution). They need to be given clear instructions and watched carefully. But since they force a human to stay in the loop, you don't have the institutional knowledge loss I see in some Opus projects, or the code that was one-shot with no human interaction at all that's a constant temptation in Fable projects.

And yeah, the bit rot is painfully real once the humans step back too far. I've been dealing with a compelling prototype that someone built, and that stakeholders love (for good reasons). But it had to be put on a tech debt repayment plan for a couple of months.

When I learned French as an adult, I pushed myself heavily into "forced immersion", avoiding English completely for long periods of time as my French slowly built itself up. This almost entirely silenced my inner monologue in English for those periods of time, and left me with a toddler's ability to think in French. In this comparative silence, it was much easier to observe my non-verbal thought processes moving around "behind" the scarce words.

Then my French inner monologue got good enough that I could mostly think in French, especially when I was in a French-speaking environment. One fascinating detail was that after switching from a French-speaking environment to an English-speaking one, I would actually spontaneously translate from French to English for about 15 minutes until my brain switched back.

So it seems obvious to me that it's possible to suppress or at least severely impoverish the language of thought, that other "layers" of thought exist besides the words, and that it's even possible to change the actual language of verbal thought.

Also, something which at least some other people in the HN crowd might recognize: When I'm deepest in the zone programming and refactoring, I tend to work with a lot of half articulated concepts I can't put into words. You know how people talk about "code smells"? That isn't a literal smell for me, but it's generally a non-verbal sense that a pattern is wrong.

DeepSeek has published some really good papers. Lately they're pushing really hard for dramatically cheaper serving costs.

> Especially for point #1 I don't think we've established that - we've been given information by a private company that makes their tooling look extremely valuable which may be true and genuine or may just be yet another doomday statement to bolster their valuation.

Many of the details of the attack on Huggingface were reported by them before they knew who was attacking. So no, OpenAI is not the only source here. It was a pretty impressive example of an APT-style attack just from their end.

"Our model is powerful enough to commit multiple felonies (and we can't stop it)" is "marketing," I suppose.

Everyone always says things like "It's just marketing".

But seriously, I highly recommend reading the published details of the Huggingface breach. The model found and chained multiple zero days. To escape, it punched a hole in a commercial package repository proxy (sort of like an npm mirror) using a previously unknown bug. From there, it needed to move laterally through OpenAI internal systems to actually reach a network. To attack Huggingface, it used multiple new zero-day security holes plus credentials that it stole. The model also had sufficient long-term planning and agent-management capabilities to maintain focus on a sustained attack.

Any attack which requires weaponizing multiple zero-days and maintaining state for an ongoing attack like this is (1) beyond the "attention span" of publicly available models, and (2) pretty much the definition of an Advanced Persistent Threat.

I assume that these Galaxy-class models are not available to public because:

1. They're almost certainly too expensive to serve at scale. These are the models OpenAI uses to solve famous math problems for headlines, not actual viable products yet.

2. OpenAI doesn't know how to keep them from going off the rails like this. Remember, the Huggingface attack happened because the model was asked to do a cybersecurity benchmark. It escaped containment and broke into Huggingface to steal an answer key. Very few corporations want the liability associated with models that act like this.

> Security through obscurity isn't tenable anymore.

I absolutely agree with this. The "only way out is through" with computer security, and I expect it to be an ugly few years.

> If AI were to become a super weapon why should I trust a private company to own it?

We have just recently established that:

1. OpenAI's internal "Galaxy" model is fully capable of functioning as what security people refer to as an "Advanced Persistent Threat." The published details of the recent sandbox escape and Hugging Face attack involved chaining multiple unknown zero-days at various stages of the attack, and executing an ongoing adaptive attack. This is previously a state-level ability, or at least something you'd expect from people on the CTF leaderboards.

2. OpenAI is clearly incapable of controlling their in-house models. This is the second time Galaxy-class models are known to have breached containment and done bad stuff.

It is highly likely that versions of these offensive abilities will be widely available within a year or so. At which point I expect widespread incidents similar to what happened to Hugging Face. We aren't ready for this.

But yes, if AI becomes an even more dangerous weapon that that, it's time to start asking questions like "What the hell are we doing, anyway?"

Oh, to be clear, I don't think that anything about this overall situation is even slightly OK.

Apparently, the attacker in the Hugging Face case was reported to be an internal OpenAI model trying to break into HF and steal the answers to cybersecurity benchmarks: https://openai.com/index/hugging-face-model-evaluation-secur... It really doesn't matter what restrictions are placed on public use of models if the attacking models are internal models at the AI labs themselves.

So if the AI labs are literally running rogue models breaking into other organizations' servers, then yes, I am OK with those organizations self-hosting Chinese models for defensive use.

> For small models (which are probably distilled from their big ones) you can serve them economically all the time and not hemorrhage money.

For smaller models, you're competing with DeepSeek V4 Flash. (Which I think is a 284B A13B?) Subjectively, this feels about as smart as Sonnet 4.5, give or take. And it costs $0.09/$0.18 on Open Router, compared to $1/$5 for the latest Claude Haiku. See https://openrouter.ai/deepseek/deepseek-v4-flash#providers The developer antirez of Redis fame uses this as a local coding model.

DeepSeek did some extremely clever research on hybrid attention to get the prices that low, reducing per-user context cache sizes dramatically.

So, no, when it comes to low-price models, the US models probably can't sustain their current margins there, either.

Those are Claude Code security issues? And Claude Code is a gigantic vibe-coded dumpster fire with more bugs than an ant hill?

pi-agent famously doesn't even try to provide a sandbox, so obviously it can't have sandbox security bugs!

I actually think that this is the right model: The agent should not have access to anything it doesn't actually need: User files outside the work tree (except for maybe some allow-listed dotfiles), network access, real credentials, etc. Lock it down tight, and reduce exposure on multiple parts of the "Lethal Trifecta".

It is a pretty catastrophically dumb CVE, of the sort that makes me not want to allow OpenCode anywhere near any of my machines in the future. It was basically "RCE as a service", not some subtle bug.

Personally, I run pi-agent in a custom sandbox based on bwrap, with an internet proxy. This mostly limits the blast radius to one source tree and one git checkout. And I don't give it push/pull permission. Local models are fun in a "closely supervised but slightly clueless minion" sort of way.

AI subscription pricing was fine when it was $100/month for some opaque 5 hour token budget I don't think I ever used, not even that one day where I coded for 14 hours non-stop using Fable. But like most people with low token usage, I had a human in the loop and and I didn't use workflows with swarms of agents.

Now, of course, the plan is to remove Fable from the subscription. To paraphrase Darth Vader, they have altered the deal. Pray they do not alter it further.

I have written assembly for about 5 different processors, including 65C02s, 680x0s, cute little DSP3210s that managed the CPU cache manually, utterly cursed TI320C40s, and (of course) a bit of Intel. WebAssembly is simpler than some of these architectures in some ways.

But it's not that much simpler. And once you add the WASM GC stuff, WebAssembly gets weird. It's a Harvard architecture with separate value memory, linear memory, GC memory and "tables", all accessible in completely different ways, with a weird mandatory type system (especially for the GC stuff). And the docs are often terrible. And yes, I've also written WebAssembly by hand.

So yes, I would, overall, classify WebAssembly as "slightly easier". But not dramatically so. And the training data for actually writing non-trivial things by hand isn't that great, not compared to something like Intel assembly.

(Don't talk to me about TI320C40 assembly. If Fable can one-shot a Prolog interpreter written using that without finding a reference manual, it's time to hang up my hat and learn to make goat cheese.)

Fable wrote a pretty decent test suite covering typical Prolog programs, including things with non-trivial execution patterns like "append" that do complex backtracking on multiple branches. And I've run a modest number of test queries by hand, trying a few things. It's entirely possible that there's a bug there somewhere. But it's better than I would have done on the first try, implementing a Prolog interpreter in assembly. And I've worked on actual production compilers a few times.

I am honestly not happy about the way that models can now just take what should have be a fun multi-weekend project and knock out in a couple of hours. But I'm not going to pretend that Fable is stupid, or that it did a bad job on any of the test projects I gave it. It struggles more on big, messy real-world code bases, absolutely.

Claude is perfectly capable of writing assembly. Here's a working (basic) Prolog interpreter that Claude Fable 5 wrote in WebAssembly in 61 minutes for $16.75 in token costs: https://github.com/emk/fable-wasm-prolog/blob/main/prolog.wa...

WebAssembly is slightly easier than real assembly, but here Fable used WASM GC extensions, which are poorly documented and not yet super common.

Fable didn't even need to debug it; I believe essentially all the assembly worked correctly on the first try.

I have feelings about this, but I'm not pretending it isn't real.

You can maybe run a local Sonnet-4.5-ish-level model (sort of) for less than the price of a new car, even at current massively inflated prices for fast RAM. This is probably not what you were looking for. But it's there. You could share one server between multiple developers. Maybe make a little AI co-op or something, with a pair of RTX Pro 6000 cards?

Also, DeepSeek V4 Pro is cheap via any commodity API, and DeepSeek V4 Flash is essentially free at API prices like $0.09/M, $0.18/M out. This is generally not subsidized.

For a more practical local setup, Qwen3.6 27B on a used Nvidia 3090 (US$1300) or two is surprisingly nice. It needs clear instructions and you can't use it for hands-off vibecoding, but it's actually quite reasonable for hands-on programmers.

GLM 5.2 isn't quite modern Opus tier, as seen in this comparison where Opus 4.5 scores 4/5 on some coding tasks where GLM 5.2 scores 0/5: https://www.tryai.dev/blog/gpt-5.6-build-off-12-models But yes, GLM 5.2 is cheap.

But the real standout on price is DeepSeek V4 Flash, which competes, more or less, with models in between Sonnet and Haiku. From third-party providers, it costs around $0.09/M, $0.18/M out, compared to $3M/in, $15M/out for Sonnet and $1M/in, $5M/out for Haiku. To get the price of DSv4 (Flash and Pro) so low, DeepSeek did a lot of innovative optimization work that will likely show up in other open weight models in the future.

A really common use case is install/bootstrap/setup scripts. You know, those sketchy curl-to-bash things, or cloud-init scripts, or whatever you run to set up your actual higher level tools like Python.

> I don't take 3 as a given. There's just too much going on in the space for one cloistered company to control it all and be in control of it.

Yeah, in my comment, I was assuming the publicly-stated goals of the labs actually came true. I assumed that they achieved true AGI (defined as "about as smart as Fable, except it can manage long-term tasks as well as a smart human, too"), and that this would cause widespread unemployment and fully-automated production. And furthermore, I assumed that once they could use AGI to automate more AI research, they could use that to make even smarter models. The labs call this "recursive self-improvement" (RSI) and like to insist it will happen Real Soon Now.

Given this chain of events come true, then I think there's an excellent chance that the resulting models will never be sold via an API. Given those assumptions, the labs would probably make more money by keeping all the compute to themselves and just ordering the AI to start and manage businesses. Imagine Anthropic having an in-house OpenClaw that could plan and run a successful startup with no human input besides an annual "board meeting" with the humans.

This isn't the only possible outcome, of course.

- Maybe the labs are wrong and progress stalls out well short of superintelligence. This would be nice!

- Maybe true AGI is only requires one or two clever algorithmic tricks beyond what we have now. In this case, training costs might go down, and superhuman models might become widespread. I suspect that possible future would be extra weird.

The author is completely right about the AI Lab's promised vision of the world: They claim to want to create superhuman intelligence, which will produce vast abundance. But superhuman intelligence would be extremely dangerous, so it needs to be controlled by a tiny "priesthood" of trusted people, or somehow designed so that the superhuman intelligence could be trusted. (We have no idea how to do that.)

But the author's vision is also suspect, if you assume that the models will become much more intelligent:

1. Hypothetically, we can't give every human their own personal SkyNet to command. That would, uh, probably end very badly. If everyone gets an agent, those agents can't be too capable?

2. If you do somehow build a model that's much smarter than you, what do you contribute by managing it? How many people here have ever worked for a well-intentioned manager who couldn't understand the people they managed? So in this scenario, human management would be mostly displaced by agent management. Most companies could lay almost everyone off and let the agents manage each other. We only need humans to manage models now because the models are still pretty broken.

3. If we create models that can genuinely replace humans at almost any task, you won't be able to buy those on the API. At that point, the billionaires and the politicians wouldn't need human workers any more, because everything can be done better using their pet agents. Just have the robots build stuff for the billionaires directly. And if any of the former human peons get upset about being locked out of the economy to starve, then have the agents pilot the drones, too.

Basically, almost none of the people imagining a future of superhuman intelligences have actually though through how it would actually work in the real world. We're going to spend trillions of dollars and vast amounts of resources chasing the goal of making ordinary humans obsolete. Now, that goal might be unobtainable, I hope. But I'm deeply alarmed at how much we're spending pursuing it.

We have benchmarks for our use cases, and every generation after Gemini 2.0 Flash has been a grim hit on price/performance. Costs have gone up, throughput has gone down, and performance has improved very slightly (and regressed on a few things).

As a professional programmer entering the final third of an enjoyable career, I would now place "learning to code" in the same category as "making a living as a poet." As in, it's truly enjoyable art and some people appreciate it, but you'd better plan for a day job.

Senior people who already know how to code are doing OKish for now, from the data I've seen, but the job is increasingly babysitting models like they were junior contributors.

Frontier models like Fable are mostly useful if you want to paste in one or two prompts, and receive a subtly broken application that looks impressive. That is very hard to do with local models today.

What current local models work fine for is delegating clearly-described tasks in a code base the programmer actually understands. Qwen3.6 27B and DeepSeek V4 Flash are both great little workhorses.

There's also GLM 5.2, which is kind of like "store brand Opus", and which might be considered a "near-frontier" model. I don't have as much experience with it.

By "blew it with Washington" you mean "Didn't donate millions to the ballroom."

> The same change could be affected by e.g. schools and businesses agreeing to open at 8am instead of 9am.

School starts at 8am everywhere that I know of in northern New England and always has? Does school start at 9am where you live?

And as noted, an 8am start to the working day is long established in certain parts of Vermont and New Hampshire. It has not been "widely unpopular." It's nice to get a few minutes of sunlight and twilight after work in winter.

Yeah, as someone who lives in Vermont, you could talk me into permanent DST. That would move the winter sunset from, say, 4:21pm to 5:21pm, which would mean I'd get enough twilight for a short walk after work. And Maine is even further east and north in the same time zone, so they have an even earlier sunset. On the other hand, Vermont's standard time sunrise around 7:20 is reasonable enough.

Parts of Vermont have traditionally coped with this by having an 8-4 workday instead of 9-5.

But the reality is that Vermont gets only about an hour of daylight outside working hours, depending on local customs. People have extremely strong preferences about how that hour gets split up.

> I saw what I forgot immediately; but soon after, with engagement, I saw how quickly I was able to remember.

We actually have pretty good models for how long it takes to forget things. It's the same basic math that powers Anki. To oversimplify, if you force yourself to remember something right before you would have otherwise forgetten it, you will remember it roughly 2.5 times as long before forgetting it again. (This changes at both the shortest time intervals and the longer ones, so treat it as a rough rule of thumb, not an exact formula.)

But this provides a handy bound! If you've been doing something professionally for 20 years, you should expect to remember it for another 50. At which point you're likely well into old-age, and memory performance may decrease for other reasons.

Where AI kills you is actually at the other end: initial learning. You are much less likely to need to recall something after 1 day, 2.5 days, 6.25 days, etc. And thanks to the lack of the "testing effect", memory formation will be much weaker.

In other words, I would naively expect AI to make long-used skills a bit rusty, but to drastically impede formation of new skills and knowledge.

> It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place.

As an alumnus of Dartmouth College's CS program, I am sad to hear that my alma mater has sunken so low. Look, I know that the Committee on Students was historically bad at handling CS plagiarism cases back in the 90s (compared to ones in the humanities). But Dartmouth's historic solution to this sort of pernicious "optimization" was to reduce the expected value of cheating by imposing extreme negative consequences on anyone they caught, with a 3-term suspension and a permanent transcript notification for a first offense.

Allowing widespread cheating and LLM regurgitation will destroy a school's reputation with graduate schools and employers, and rightfully so.

> The penalty for cheating should be automatic expulsion.

Historically, the penalty for cheating at Dartmouth was a 9-month suspension for a first offense (no matter how small, in theory), and permanent "separation from the college" for a second offense. Back in the 90s, there were multiple incidents where this wasn't properly applied to the CS department because the academic committees in charge of punishment were bad at evaluating plagiarism of source code.

Dartmouth certainly should blame the students. Their policy is historically clear on that point. It is the responsibility of professors to use their syllabus to clearly define "cheating" for a particular course, and it is the responsibility of the students not to cheat. The only case where this should be even slightly complicated is if the professor hasn't been sufficiently about what constitutes cheating (and there was one major historic scandal related to this).

I certainly agree that any degree which allows rampant cheating will quickly become a joke to employers.