Comment by ben_w | Hacker News Reader

Comment by ben_w | Hacker News Reader

Parent Full thread

Anyone saying 10x as a serious claim is clearly using a round number and vibes; however, even if it were so, AI getting popular 3 years ago does not mean what you say.

The earliest used models in this category would easily 10x (what did I just say) the creation of one-off short scripts, but you're not writing a 30-year app out of just a bunch of short scripts.

The METR time horizons graph suggests we can now get 10x (ahem) speedup on solving most coding problems that take us a few hours and about half of problems that take 2 days. Amdahl's law bites: even infinity speedup on half your problems is only 2x overall.

If you let an LLM loose, with a huge budget, what's the biggest artefact it can make before it drowns under the weight of bad decisions? The C compiler and web browser headlines a while back? The maths papers we see now that solve problems which stumped the maths world for decades?

But this is the other side of the same coin: teamwork. One person getting a thing made in twelve months vs a team of a hundred, you can scale up fast with money when you have a proof of concept, and an LLM can make a lot of proofs of a lot of concepts even in free accounts.

LLM-assisted software engineering seems to be very efficient if you have a deterministic target. (bun rewrite from zig to Rust, 100% Node.js compatibility, pnpm compatibility -- https://github.com/oven-sh/bun/pull/38333)

My pushback on that would be that all of the "engineering" concering a rewrite was already done the first time around. It's a glorified translation.

I do agree that the port would've taken a lot longer without LLMs though.

Problem with that is that it's not proper rust code, it's some kind of franken zig style rust port.

The actual hard part is getting it into idiomatic, safe rust, and I don't believe the LLM port makes the full transition any easier than doing things the old fashioned way: a dual lang code base like Linux.

And compiler generated assembly code is ugly and spaghetti, unlike beautiful hand-crafted human written assembly code, the kind you see in ffmpeg codecs, ...

Doesn't matter.

It does matter, because that's the entire point of the language. You know, memory safety?

The vibe coded rust port did not get them any closer to full compiler verified memory safety. They still need to go file by file, bit by bit, and make it memory safe, at which point, why not just do that from zig.

> at which point, why not just do that from zig.

because for the same effort you get a better result if you can and do use a better tool?

I didn't say don't use LLMs. In fact, I'm quite certain they could help quite a bit.

But I don't think they did it in a way that actually provided a meaningful gain. My point is that the state of the codebase right after the code is just a worse version of the original zig code with few of the rust advantages. Going from that Frankenstein rust code to actual memory safe rust is a similar leap to going from zig.

> I don't believe the LLM port makes the full transition any easier than doing things the old fashioned way: a dual lang code base like Linux

...

how so?

let's say Linus opens a branch, rust-temp, and in 2 weeks pushes ~15 million lines of code deleting most of the old C code. and then it gets merged in a few weeks. and then there's still a few months of the "merge window" and RC process. and each day folks report bugs, and automatic fuzzers make sure that both versions "behave the same".

I was pretty skeptical (still am), but what the Bun team did is pretty great so far.

https://bun.com/blog/bun-in-rust details the process, we see the results (Node.js compat [0], more than 3 thousand of issues fixed since since 1.3 [1], more than 900 issues fixed in ~24 hours [2], and it's live/in-prod [3])

> 50 dynamic workflows in Claude Code run continuously over the course of 11 days.

so let's say 500 workflows could do something similar for the kernel. the reported cost is 165K USD, so let's say this would cost 1.65M USD, plus CI costs [4] (which is ~200K/year for bun, so let's say it's 2M USD for the kernel)

How much the world is spending on kernel bug bounties and various security programs each year? (rough estimate says that just the visible kernel testing programs cost at least 15-30M / year)

The bar is not perfect. The bar is something better.

[0] https://xcancel.com/bunjavascript/status/2087436767054213517...

[1] https://xcancel.com/bunjavascript/status/2082298681223680002...

[2] https://xcancel.com/bunjavascript/status/2080882189663907859...

[3] https://news.ycombinator.com/item?id=48966569

[4] https://news.ycombinator.com/item?id=49070494

Because they are still dealing with memory safety issues to this day. They ported it to rust, completely introduced a bunch of bugs, went through all that effort, just for the codebase to be not much safer.

I'm actually basing my opinion off the blog post, specifically the code. While they do get a few changes for free with the port, most of the code looks identical. And the number of unsafe statements certainly agrees.

And now they're going little by little, playing whack a mole with seg faults... My point is that the step they're at right here is the important part, and that mechanically converting the entire codebase to rust was an optional step when incrementally rewriting would've worked just as well if not better.

Being "only" 165k USD doesn't mean anything, because you didn't do a complete port, you did a port in a trench coat. If they went for the more targeted module by module approach, and did a clean and proper rewrite, LLM assisted or not, they would reach their end goal much less turbulently and without the effort of that initial frankenport.

are they dealing with more or less memory safety bugs?

for me it was already "don't run in prod" quality before. (at least after this I'm considering adding it to the CI to see how it fares.)

segfaults. I don't know. I looked at their CI and GH issues over the past few weeks. (though now GH is down so I can't do a search, but I didn't see thousands of segfaults.)

by all accounts and measures it seems it made their house of cards more manageable. despite the frankenport, no?

they already had a zig compiler fork, wanted to upstream it, but the zig maintainer(s) said it's low-effort. now they don't have to maintain their compiler fork. (or wait for the zig team to deliver the features they wish for.) no need to maintain a hybrid codebase. (though it has C++ because of the embedded JSC.)

also I have no idea what's the zig-rust FFI status, but getting over with a rewrite faster is usually better, even if you are left with non-idiomatic code.

Great example. There is no way I would run that in production anywhere.

Long term stability is the thing that matters, such a move would be immensely threatening to long term stability.

the neat part is that you don't have to, millions will do it for you!

the point is that it's a quite obvious opportunity that was just some pipedream years ago.

by the time v8.0 comes around we'll have more data on the bun rewrite. also we'll see how LLMs will affect the productivity of kernel developers, and the overall stability/maintainability.

> the neat part is that you don't have to, millions will do it for you!

You're not wrong. We're already seeing something similar with all the outages and buggy software updates these days.

We little choice in the matter, you will have ze (software) bugs!

And how long will it take to rewrite the Rust rewrite to idiomatic safe Rust?

Reply on news.ycombinator.com