lunar_mycroft

It's funny, because editor choice is also an analogy I use, to argue for the exact opposite conclusion.

Your hypothetical developer wouldn't be using notepad because they're unaware of other editors, they'd be using it because they evaluated other editors and concluded that, for whatever reason, they would be worse for them. I'd be fascinated to hear why they came to that conclusion, but I'm not going to tell them they're wrong if they're performing acceptably, aren't constantly breaking CI because the linter rejects their code, etc. Everyone is different, and I'm not narcissistic enough to think the fact that I would be way less productive without my modal editor, LSP, linter, terminal multiplexer, etc. justifies forcing everyone else has to adopt my exact setup.

You can set the lints to `forbid` instead of `deny`, which means they can't be `allowed` like that.

Normal users aren't using harnesses in the sense developers think of them. They're interacting with models where they've been shoehorned in for no good reason, or they're using them nearly entirely through chat interfaces.

Is Claude really that much better than all the others for normal use?

(Sorry I'm being vague, but I'm not sure I'm not sure what's public knowledge)

The cap is moderately above the high subscription tiers, and managements/the executives were clearly extremely concerned about how expensive it would be if we all or even mostly came close to hitting it. I heard that they originally wanted to go lower but the developers in the pilot program blew past their planned limit very quickly.

As for the company, its almost entirely B2B SaaS (I think it has some offerings that are used by consumers, but they're mostly/entirely paid for by another business on behalf of their customers), and they have developers all over the world, although the headquarters and biggest group of developers is in silicon valley (my office is in the midwest).

Uber is not the only company that's putting a per-developer limit on AI spending. I know this because I work for another one (and we have a significantly lower limit). You just heard about Uber first because they're high profile.

I'm not sure I'd classify "double" as "the same ballpark", but either way C2Rust appears to just mark everything as `unsafe` (even when it's completely unnecessary), so I don't think that's a fair comparison. Further, I know for a fact that their are instances of the `unsafe` in the bun codebase right now that are just trivially incorrect in rust. A vaguely competent human programmer, even one doing a 1:1 translation (which is likely a bad idea anyway) would have caught those.

Yes and no. In theory, you could start to go through an factor out unnecessary unsafe blocks from the codebase now. In practice, writing safe rust often requires significantly different design decisions than writing in garbage collected or fully manually memory managed languages, especially if you want the results to be performant. There's a very good chance you'd be better off rewriting from scratch than trying to do a 1:1 translation/port.

Bun has about twice the density of `unsafe` compared to deno, which does roughly the same job (wrap a c/c++ javascript engine to make a server side runtime written in rust). So not as massive a difference as the linked post's comparison, but still significantly more unsafe than we'd expect.

> Comparing unsafe counts to UV makes no sense here.

Agreed. The better comparison is Deno, which does roughly the same thing (wrapping a c/c++ javascript engine to create a serverside runtime). Deno was smaller, but per line of code/file it had just over half the unsafe as the bun slop rewrite. There are also examples of unsafe blocks that are just locally incorrect [0], meaning you don't even need to read outside the function and it's definition to determine that the entity doing the porting (in this case claude) didn't understand how unsafe rust works.

> I am skeptical why Bun was even an acquisition target in the first place, other than pulling stunts like this.

From what I've heard, it sounds like the author had become fully hooked by LLM coding before the acquisition, so I don't think we know enough to conclude that this was premeditated by Anthropic. I think the more likely trigger is that Bun had just bragged about how they'd forked the zig language for a performance increase and attacked zig for it's no-AI policy, only for a zig core team member to point out that the code couldn't be upstreamed even if it had been hand written [1].

The better point, IMO, is how this is an example of how AI use/addiction makes developers who fall into it loose the ability to judge code quality, particularly for code they prompted for themselves. From what I've heard of Mr. Sumner and the team at oven from before AI, I very much doubt code of this poor quality would have been allowed anywhere near the main branch before LLMs.

[0] https://github.com/oven-sh/bun/blob/d2a6506dfad4c3ef3dddb9ae...

[1] https://ziggit.dev/t/bun-s-zig-fork-got-4x-faster-compilatio...

First, you're very likely underestimating how much load SQLite can handle. SQLite is usually write limited, but for smallish writes it can easily handle thousands a second with very trivial optimization, and with more thought can scale to tens or hundreds of thousands of write transactions per second. In some cases, it can actually out perform traditional server based RDMSs because of reduced overhead and because holding locks on network timescales (which will likely happen even for databases with multiple writers, because eventually you have to deal with two transactions needing to write to the same place) is very inefficient.

Second, I think you're overestimating how hard sharding is here. There are plenty of use cases for which sharding isn't just easy to set up, but the natural thing you'd be likely to do even without scale. Things like e.g. a helpdesk SaaS, where each customer/organization has it's own independent data.

Third, a large part of the point is that you are unlikely to know ahead of time you're going to "scale like that". As I already pointed out, most SaaS apps do not end up having many users. For some that's intentional, but for others the reason is that they simply never caught on. For those cases (and they're the vast majority), cosplaying as a much larger app is a complete waste. It's much better to wait until you're successful enough to need to switch and then use the revenue you now have to solve the scaling problem you ran into.

Fourth, as an aside, ironically the other (slightly less) easy way to shard superficially resembles a common way people cosplay as netflix: splitting your data by domain as "microservices" (although there's a good chance they don't need to be independent processes/on independent machines).

I didn't say they did. What I said was that if the person I was replying to was correct that average Elo ratings had increased because of AI, the mechanism would have to be "AI makes lower ranked players give up" instead of "AI makes players better on average"

Two possibilities:

1. Your unit tests are exacting enough to fully specify the unit. In that case, congratulations, your unit tests are the code. They're also probably much more awkward to write, maintain, etc. Also, the compilation step to go from the unit tests to the actual code is now orders of magnitude more expensive, requires a SaaS to even work, etc.

2. Your unit tests are not that exacting and still leave ambiguity, edge cases, etc. In that case it very much matters what's in the blob of code, because while it could be a correct implementation of what you wanted, it could also be something else entirely that just happens to be correct for the part you did specify.

Right, but you have to give the new players some rating, and as long as that number remains constant it will also be the mean Elo. Therefore, "new players entering the pool" can't cause a rise in mean Elo. As for lower rated players leaving, that could indeed raise the mean (assuming you stop counting them in the average), but that changes the point from "players have gotten better because of AI" to "worse players are more likely to just give up on chess because of AI", which is a significantly less optimistic picture IMO.

Elo is zero sum. Each point gained by one player is lost by another. It follows that the mean elo is always exactly equal to the initial elo assigned to new players before they play any games, and can't change over time. If the highest ranked player are higher elo than before, the lower ranked players must be lower elo.

> This February I tried again and used Claude to generate Rust code. I have never been more stunned in my life. It's just as good as I am, and 30x faster. No fluff, the code is verbatim just as I would have written.

Having looked at a bunch of known or suspected (based on the intent of the code and/or what I know about the developer(s)) LLM generated rust, there's only a few explanations here:

1. You're way better at prompting than (virtually) anyone else.

2. You're vastly overestimating how good the rust code it produced is.

3. You handheld the model throughout and made lots of edits.

4. Your hand written rust code is very bad.

Because from every example I've seen, these models write horrible rust. Sure, it may technically pass all the tests, but it's horribly pessimized, badly organized, doesn't even attempt to use the type system, if there aren't bugs now there will be the second it tries to refactor or add a new feature, etc. etc.

(I also strongly suspect that the same would be true for other languages, but I can detect it in rust more easily because it's my main language)

The typical multi-user SaaS webapp doesn't have anywhere near enough users to overwhelm a single SQLite instance. Of the few that do succeed to the point where that's no longer true, a significant fraction can use techniques like sharding to stretch SQLite further.

SpaceX is "in the process" of a lot of things, not all of which pan out. So far the cases that have actually started serious construction are in Cape Canaveral, and will absolutely be necessary assuming Starship becomes operational (because the number of launches SpaceX is allowed to do from Starbase is limited).

Launch pads are not something you just buy on a whim to keep around just in case you need them. They're very expensive pieces of infrastructure that you only acquire when you have an actual, known need. That's how every launch provider that I know of behaves, including SpaceX.

> we've had say superhuman competitive programming performance for awhile

Fair. Question though, is this when compared to competitive programmers, or developers in general?

> extremely strong performance (super-p90-engineer) on say language-to-language porting

I'd need to see the methodology here and could easily be wrong, but I suspect this is largely down to "faster" and "willing to do a lot more of it without complaining"

> RE-bench (ML research engineering benchmark from METR) is already clearly above human perf

This pretty much has to be "relative to devs who don't specialize in that area", because if it wasn't the frontier labs wouldn't be paying a fortune to hire ML researchers.

> Mythos clearly has superior cyber capabilities

Based on Daniel Stenberg's experience with it [0], it seems like it's at best roughly on par with human experts. It's advantage is cost/speed.

> Also, why do you discount speed and cost so much?

Because in all the domains LLMs are applicable to, getting something cheaper/faster at the expense of quality isn't new or particularly interesting.

> Every model iteration is a significant bump up in performance according to a lot of complementary and principled measurements. What's been the thing that hasn't been true?

That they were good enough. To reuse the baby analogy, if every week your friend told you that their infant child was now heavier than an elephant (while acknowledging that the baby was lighter than one the previous week), and every week that turned out not to be true, it wouldn't be a defense of your friend to argue "ah, but the baby was heavier every week than the week before".

Also worth noting that as of ~8 months ago, while benchmark scores were steadily increasing, merge rates (aka whether the code was "good enough") were not [1].

> thats a bit of a crazy ask.

Why? If you use LLMs to do anything you're basically doing that already, it's just that the scope of your Y is smaller. Either the benchmarks are irrelevant and you're using something else to determine when that's appropriate for a given Y, or you do in fact have a value of X for the Y's you've handed over to LLMs.

> Yet one side has a mountain of hard evidence and the other side has...an outdated n < 20 METR study using Sonnet 3?

There's a lot of irony here, because by far the most common pro-LLM coding argument is "I feel like I'm producing good code faster with them", followed by "this other person feels like they're producing good code faster with them".

Also note that the most important part of the METR study you reference wasn't the slowdown they observed, it was the dramatic disagreement between what the participants thought the impact of AI was vs what it actually was. That isn't dependent on the model.

[0] https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...

[1] https://entropicthoughts.com/no-swe-bench-improvement

Until recently, SpaceX only acquired new pads because they needed a completely new launch site (SLC-4 in Vandenberg) or needed to launch a vehicle that their existing pad(s) didn't support (Falcon Heavy for LC-39A, Starship for Pad A in Boca Chica/Starbase). Currently, Blue Origin's only orbital launch vehicle is New Glenn, and their Vandenberg pad is still under construction.

No, this would be crushing regardless. Even if Blue Origin had dozens of rockets ready to go, they can't fly without without the pad, which will take around a year to repair (based on previous examples).

First, to clarify my own position here: I use LLMs for code review, to help with some planning, for the occasional throwaway prototype, and as a more advanced rubber duck, but I do not let LLMs write code I care about, even with human review (because human review is imperfect).

> its uneven... if its saying it now clearly has superior abilities in many parts of the SWE workflow then its demonstrably right.

In what ways are current LLMs better excluding speed and cost (because getting those things by relaxing constraints on quality has always been trivially possible)? Even the fabled (heh) Mythos seems to be at best roughly equivalent to a competent human security researcher.

> I think you are saying "people keep saying it's good enough to replace SWEs(?)" every six months and they are wrong every time. I don't disagree that we have not gotten to a "we dont need SWEs anymore" point, but I think its a bit of a strawman: who is making the claim you are addressing?

Most of them aren't saying that the models are good enough to full replace developers, but this definitely isn't a strawman. I've been seeing the same basic claim for at least 18 months at this point.

> No no, I'm talking about performance in absolute terms. These are strong proxies (SWE-bench, etc)

Unless you have some non-LLM scores to compare to, those are still relative measures. They show/suggest that LLMs are getting better (at least in some ways), but without a definition of "good enough" in the same metric, that isn't sufficient to say whether or not they are.

> No but then the point I'm making is we're drifting further and further away from Occam's razor.

Both sides of the debate have to explain the fact that a lot of developers disagree with them, so I don't think this argument really works.

Worth noting that the person you're replying to is not the same as the one who said frontier models now outperform developers.

> you’re saying we only have relative performance numbers and not absolute measures of capabilities and reliability but that’s simply not true.

No, I'm saying that the claim you were making ("current models are better than some non-model based standard X") does not follow from your premise ("current models are better than past models"). It's possible that your claim is still true (although I don't think it is for most of the values of X that matter), but that wouldn't change the fact that the argument made is invalid.

As stated, your argument was basically the classic "my 3-month-old is now twice the size he was when he was born" meme, except if the tweet claimed that the kid currently out weighed an elephant.

> That’s the inflection point.

No, it isn't. An inflection point is when the direction of curvature changes. If we crossed over into the diminishing returns part of the logistic function, that would be an inflection point (as would the case where we had been in the diminishing returns regime, but then progress went back to speeding up).

> Implication is a massive jump in adoption.

The point I made was that "a massive jump in adoption" doesn't actually imply "the models are actually good enough now", only that a lot more people think they are.

> Are you aware of performance trends though? You’re painting a picture that seems to ignore how things have consistently trended for many years now, even pre ChatGPT.

Models have been getting better, but all that follows from that is that newer models tend to be better than older ones. It doesn't follow that they have (or even will in the future) gotten better than anything else, be that human developers, a given definition of good enough, etc.

> It is absolutely data driven to say “an inflection point has happened within the last 6 months”.

With all due respect to OP (who I think is responsible for popularizing that way of phrasing it), I don't think it is when you consider the actual definition of "inflection point". At best I think you can say that models crossed a lot of developers definition of good enough around then, which is a different thing. The problem I have with that is that as a (mostly) outsider looking in, it doesn't seem like they're right.

Maybe, but that's a different claim than the one I was responding to. And also raises the question of "if the lower quality but cheaper output of frontier models is more cost effective than humans, is the even lower quality but even cheaper output of OSS models is more cost effective still?" With an absolute rule like GP suggested ("no, you always want the best code generator") the answer is clear, but it get much murkier if you reject such rules (as you have to to be an LLM coding proponent)

I've seen the code they produce without extensive help from human developers, this is clearly false.

Good to see the classic "yeah the models weren't good enough six months ago, but this time they actually are, promise! Please forget you were hearing the exact same thing six months ago!" is alive and well though.

> For coding you always want to go with the best model in the category

This is transparently false, because the best "model" is still competent human developers. They're just more expensive. If you're willing to use current LLMs at all, it means you're willing to sacrifice quality for a better price, and your disagreement with the comment you were replying to is entirely about what the optimum tradeoff is.

You're basically positing that the real cost of a 20% headcount increase is higher and/or the productivity gain is is lower than 20%. That isn't an unreasonable claim, but it's basically rejecting the premise here. You might just as well object to the premise that you can buy a 20% speedup by spending an extra 20% on tokens.