It is impossible to review the entire rewritten codebase. There are just too many lines of code, 1 million lines to be exact [1].
There are countless reasons that this is a very bad thing for consumers of Bun. First and foremost you've instantly lost any and all assurance that it works the way it's supposed to. Every project has load-bearing bugs that may or may not still exist. Can you trust the core behavior is the same?
If the Bun maintainers are willing to completely replace their core product over the span of a week, how is anyone supposed to rely on it? What's preventing another rewrite in a few more weeks?
If you want to position your product as a dependency, it needs to be stable and reliable. Throwing away a million lines of code and replacing the entire product overnight is pretty much the polar opposite of stable and reliable.
It has nothing to do with AI rewriting the code, it's the reckless abandon and wild disregard for consumers that is the problem. It's literally a rug pull.
It’s all the same just different syntax. Which, by the way, is why it looks ugly to rust developers. The devs wanted the code to look familiar to them.
I do think they should have called this 2.0 though. Would not feel such a rush (1.3.14 has a few regressions, and no one really cares because there are lots of small rust fires now).
Overall, the bigger issue is that bun chases shiny objects. But never finishes. Just look at test stuff. Most of vistest, but not all. Most of jest, but not all. Most of pnpm, but not all. Now we have image stuff, so most of sharp, but not all. dev server? Most of vite, but you guessed it… not all. Long running process… mostly like node but with memory leaks (and a motivation for rust I’m sure).
When I saw them posting about the Image routines my heart sank. Another shiny object. Coincided with test bugs so I moved to vitest completely.
That reminds me of Chris Reigrut's story from https://thedailywtf.com/articles/Holiday_Smorgasbord
Thanks for the chuckles!
>>> log.fatal("Aaaaarrrgghhhh");
Reminds me of working at a small Japanese company and someone had written a try-catch for retrieving a crucial bit of config that just logged "X missing. This is disuster." and terminated.
As lots of large and small companies have shown, test suites can only find what you test for. Vibe coded test suites can find?
It does a reasonable job. Its also pretty good at writing regression tests when it fixes a bug.
Where LLMs struggle - or at least where claude struggles - is fixing the actual bugs. Its very good at getting the test suite to pass. But it cheats. It'll sometimes disable a test, or do some hacky workaround that makes the test pass that doesn't fix the underlying issue. It'll say "All done, the tests pass". But sometimes you really wish they didn't.
I'm wondering if it might be better to set up 2 agents adversarially for bug hunting. Give one agent the goal of finding as many bugs as possible (via tests and other techniques). And another agent has the goal of fixing the bugs.
I’ve tried all sorts of things to keep Claude from cheating, but the only one that works is to restrict access to the tests files, which obviously isn’t a real solution.
We recently had an “AI week” at work and I spent $1000 in tokens trying out different iterations of this.
With quite a peculiar set of supported formats different between operating systems.
What if there was some malicious code within the 1 million lines?
The problem is trust... About nobody has the time and the resources to read 1 million lines of code, keep everything in their head and derive whether there are problems...
1. Are we positive the AI didn't cheat when getting some or most of them to pass? Are we sure it didn't just cheat on even 1 percent of them?
2. Are we positive that the AI didn't introduce bugs that the tests didn't think about because there was no need to think about that before (because actual people understood the code)? And therefore, those things go untested, and nobody can write good competent tests for them because nobody understands the code in the first place.
It does matter, that's why those people quit because it's such a shitshow, progress happens at a glacial pace, more and more defects and slowdowns keep being created even if they have a big QA department/teams and the users are probably trapped because the software is the only thing in town, the bosses are the ones that makes the purchase decisions, or the it comes attached to big and/or expensive machines and they can't just buy another one for another X years.
It is not immediate process, but it is a thing.
The code base itself has never and will never matter in the big picture
Clearing my throat: I am the first person to tell everyone on the team (repeatedly, until they are sick of hearing it) that the users, use cases, and organizational objectives are always more important than the technology.
But, in "the big picture" - the Linux codebase doesn't matter? The codebase that powers AWS doesn't matter? Hell, the Microsoft Office codebase doesn't matter? Look at what's happening to Windows when they treat it like the codebase doesn't matter.
For a tech org, the codebase is the reification of all of your objectives, all of your knowledge about your users and use cases and processes. Long term, a mature codebase plus people who understand it is one of the most valuable things you have. When orgs don't realize this, when they treat their workers and their work product as disposable commodities, we call this "enshittification."
Large LLM-written code is called slop for a reason. It's hard to understand because oftentimes it does not follow human logic.
Not really. At some point the technical debt accumulates and the only option is to trash it all and start over.
The only party that profits here are the cloud token providers.
Isn't this suppose to be the most advanced model ever and you're telling me they can't just schedule a cron job that detects and repairs the zig version?
Really? Did they just completely admit that the great AI future can't secure a significant project repository?
The challenge is: They already were decoupled. They've forked Zig. If your desire is to make a bet on generated code and reasoning as the entire future of software development, as Jarred and Bun have; why would you chain yourself to the crazy political nightmare that is the Rust project, when you've already got a language & runtime that's treated you well, is forked, your contributors know well, etc? Literally: Why can't these issues be fixed within the context of BunZig? Why don't they want to own the whole stack?
I feel this argument is not valid, especially for large code bases.
Documentation and code quality is what is important, not who wrote the code.
More importantly, it's not the same thing at all. All the code in windows (at least until recently) was written by humans, understood by humans and reviewed by humans. And that code has stood the test of time, proven its value and stability in the wild, on billions of systems. The fact that the current maintainers haven't needed to understand or replace the code is some indication of the code's quality.
Almost none of Bun's rust code has been even seen by a human, and it's only about two weeks old.
I'm somewhat willing to accept vibe-coded code if it's either absolutely non-critical, well reviewed, or maybe in the long term if it's proven itself. But not two week old code.
But I think the comparison is closer than you are making it sound. I sincerely doubt the Windows codebase was all written by humans, let alone reviewed. And my understanding is that the code is being regularly rewritten and replaced because of how flawed it is, it's just a massive undertaking.
Also if you look at their investment in AI-driven code rewriting into Rust, my bet would be that some modern Windows code itself is being vibe-coded.
All of which was battle-tested on millions, if not billions, of devices over 40 years. The new Bun is effectively a different project than it was a month ago, with next to no prod use.
I have no problem having a dependency on a 40 year/billions of use software, I do have misgivings about a dependency on a project that has never been used in prod, and was only written last week.
It is far easier to understand some part of the various NT source code leaks than it is to understand Claude code leak
Well, they didn't really need to. A complete rewrite is effectively a different project. You may feel comfortable using a new project in prod, but most people are not.
Project A: used in production for 3 years - high trust.
Project B: Has yet to be used in production - low trust.
IDGAF about automated tests, let other users shake out the inevitable bugs that show up in prod and after a few years of stability, then we'll see.
To me, it's not about whether humans reviewed the code or not (they didn't), it's more about "here's this brand-new shiny codebase of ~1m Sloc, of which exactly zero lines has been used in prod".
To create a PR, I need to test it manually. But the only true test is to survive production usage for a while. That’s where edge cases become normal occurrences.
10 engineers each reviewing 5,000 LoC a day for 20 days can do it.
And that is being highly conservative with the estimate. A good chunk of the the code is probably highly trivial boilerplate one can easily skim over in minutes.
You'd be just looking at both files, side by side, to make sure nothing was lost in translation or newly introduced.
Counterpoint: I look back at code I wrote a few years ago and just take it on faith that I knew what I was doing at some point. That's still better than never knowing, but it requires faith--faith in a human, vs. faith in an LLM.
In old days we chose between Turbo/Borland C, Quick C and GCC. We didn't think them same or trust blindly even if we didn't know how they worked.
The best developers hand optimized assembly for sub routines which they knew compilers were not good at, the rest of us sure didn't understand how any of it worked, but nonetheless felt the differences and chose with dollars and usage .
If you have enough tests at what point does AI rewriting software in a different language become close enough to ‘deterministic’? Maybe never, maybe not.
This is a massive codebase created within a week or so ago, no one can say what it will do till it does.
I still think it's mad, but not quite as mad as you might first think from the headlines.
Who wants to review a brand new codebase for free?
With zero reviews and no one even reading the code, there's zero confidence. For all we know, the new Bun could contain a change that causes JS file reads to return a novel for a very specific filename, and the tests people keep pointing to, intended to downplay the malpractice, would never catch it. Tests cannot cover every contrived scenario that normal human beings would never think of writing.