derdi

Just in case the above is not considered constructive enough, how about this: In my experience, with some prompting and iterations, AIs do reasonably well at writing solid, "boring" code. They do really badly at writing useful documentation, and this is a major pain point when reviewing their output. A system that claims to be oriented towards human review must address this somehow. It must talk about how it addresses it, and it must demonstrate that its own documentation is significantly better than unfiltered LLM output.

This system does not demonstrate this. It needs clear guidelines: Start with the big picture. Show an example. Show comments in that example. Don't get bogged down in the details of the internal workings of the compiler. Don't use jargon like "kernel" to mean "lowered intermediate representation". Don't use "carrier" when you mean "file". It's just a file, FFS. But again, don't tell me at all about the fact that there is an intermediate representation; that's not "big picture", and anyone who cares about compiler internals will already have guessed that you have an intermediate representation. Do tell me about the effect system in the first two paragraphs! That seems to be what sets this language apart. Do give me a brief argument why an effect system is a good match for a system where humans don't have to type everything by hand. And so on and so forth.

> a C-emitting native AOT backend that currently compiles the kernel .jqd carrier

This human reviewer gave up at line 4 of the Readme. What is the kernel? What does the carrier carry? Why does it need to be carried? Where does the .jqd come from?

Also, for whatever reason, from the submission here:

> I had AI analyze the ASTs of several mainstream languages plus a few of the conceptually groundbreaking but esoteric ones (listed in the README)

No, they are not listed in the Readme. Why lie to us right off the bat?

I like the general idea, but the human responsible for this seems to demonstrate no interest in actually reviewing their system's output. Which does not bode well.

OK, I see your point. I think there are (at least!) two categories of content creators here: On the one hand, those that consciously want to generate content spam. On the other hand, those that actually want to do "cool technical stuff" that was previously impossible for most people working alone with limited time on their hands (develop your own programming language etc.). As I argue at https://news.ycombinator.com/item?id=48892642, I think there is really a part of the latter group that are simply unable to recognize LLM-speak as bad style. It's how the LLM speaks to them, it doesn't bother them; since the all-knowing LLM does it, it must be fine. I don't know. I don't understand these people, but I believe they exist.

For example, consider this project where I also complained about atrocious writing: https://news.ycombinator.com/item?id=48876506. I think this is really just someone who wanted to "create" something, I don't think this is in the same category as recipe/blog spam. I don't think the person who put this online thinks that this landing page is worse than any other programming language website.

A little detail to your extra detail :-) I agree that there's a general "it'll do", motivated by just pushing something, anything, out there. But I think that's party because they don't know better. The people who generate the articles that reach the HN front page surely interact a lot with their agents. Their agents will certainly talk to them in "AI-shaped", "LLM-marker-bearing" bot-speak, but they apparently don't mind, because they can't tell that this is bad writing. If they did, if this annoyed them as much as it annoys many of us, they would surely adopt some style guidelines for their agents. And those should apply to their blog posts too.

As for the rating system, we already have many people who first check the comments for things like "AI, don't bother". I think that's OK; it could be better by quoting two or three sample sentences so that others could quickly judge how bad it is. I don't think it needs to be more formal than this.

I don't know, I wasn't saying that there was an upside. Sorry if my post was unclear. My "I don't think this is true" was in response to the assertion that people who use AI would be afraid of losing credibility from labeling.

> Nobody wants to label their stuff as AI generated because they removed credibility.

I don't think this is true. I think most of the people who post LLM content without editing and without even style guidelines are simply of the opinion that the resulting text is genuinely of an acceptable quality.

Agreed with the rest of your post. We should think of this in terms of quality of the result, not the process.

The docs could use some cleanup, by which I mean "throw away all the stupid jargon". How does one publish this homepage with the term "width-bearing" on it? This is not a term of the trade. It's not even "little known" or "niche", it has literally zero Google hits outside this project's documentation. (This post will be the first one.) "Fixed-width type" is right there.

The term "extensions" is also weird. It seems to be used for two things: One, "libraries" (any standard types and functions they wanted to include but don't inherit directly from the CLR's library). Two, a "Go-shaped" (*sigh*) "extension" that is both a "library" (some trivial functions like `len`) and also changes the language by adding channels and some new syntax! Wow, syntax can be imported with an import statement! But only this specific syntax, with this specific import!

And this is "opt-in", and an extension. But... it's right there. It ships with the language. It is in the language. That's not what an extension is to normal humans. And opt-in, sure. I guess I wouldn't have to use it. Just like every other language feature I don't feel like using?

Neither use as an implementation language nor syntactic similarities make one language a dialect of another. Consider that Java is commonly implemented in C++, and its syntax was designed to be easy to learn for C++ users. Yet nobody calls it a dialect of C++, not even when squinting really hard.

Erlang's computation model is functional, like Lisp, unlike Prolog.

They might want to make their docs less confusing. https://query.mt/showcase/gpu-sharing/ says "no API key needed", and talks about running llama.cpp on a friend's GPU. No API keys are needed for this. But then the page goes on about who "holds" API keys, and where API keys "stay", where API keys are "locked", etc.

> Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.

Not sure what you mean. The system I outlined is one where some "state" resides outside the process in a separate server. You don't need to serialize that, you just need to serialize the information to need to reconnect.

And my first point is even more relevant the more complex/distributed/brittle you make the whole thing: The more important it is for some specific process to stay alive no matter what, the less you want to live-slop code into it.

(Edit: Yes I'm aware of the live-patched space probe story. Human live-patching is not the same as letting an LLM try to one-shot the correct patch.)

There is no good reason for running this in-process. The maximally inefficient Fibonacci function in the article is a good demonstration why: Call it with a slightly larger number, and your agent slows to a crawl, with no way to enforce a timeout. Call it with a slightly larger number yet, and you might bring your agent down entirely, with no way to recover.

There is a case to be made for a dynamically evolving "tool server", but it should be a separate process. That would be more flexible for other use cases too. For example, multiple independent agent processes could talk to one shared tool server. Like a blackboard system, more classic AI!

And if you really do want to evolve the agent itself: As the article observes, its entire state can be serialized. Nothing is gained from hanging on to one particular agent process. Serialize its state, ask the tool server to kill it, rewrite its code, then start the new version and replay the state.

It doesn't matter if a human or a machine wrote it, it's just unpleasant to read. I don't know what the OP's motivation for their post was, but I think it's reasonable to point out that something is harder to read than it should be.

What's the fuzzing story? If this is meant to be a product, hopefully it's being fuzzed 24/7, using multiple state-of-the-art JavaScript fuzzers, against multiple state-of-the-art JavaScript engines?

Is "loop" a common term for... whatever it's used for here?

Anyway, this looks like it might be a fun way to gamify TODO lists. Would be interesting to see some output after some goals have been achieved. Screenshots of the web UI maybe?

This is a good list. Other reasons are that the JIT has more information about the program. The Java-to-bytecode compiler never sees the whole application, and it knows nothing about which program paths are actually executed in practice. The JIT compiler, in contrast, knows about the loaded class hierarchy, and it has access to type and branch profile information. This enables the JIT to do aggressive inlining of method calls and pruning of branches that are never executed. These are probably the most important transformations that lead to the discovery of opportunities for these mask optimizations. The source code that javac sees will contain very few relevant cases.

Again, I'm not criticizing that person specifically. They repeated a word that appears in all the articles.

There are two points I was trying to make:

1. At the relevant time, there were no separate Eastern and Western empires, so this city was just Roman, without qualification.

2. Even later, it could just be called Eastern Roman. Calling it Byzantine instead may be a conscious choice to make the Roman-ness disappear. More likely it's just deeply implanted, based on an education that was based on making the Roman-ness disappear. Again, I'm speaking about the general use of the term in all the articles, not by the OP.

Anyway. Even the person in this thread arguing most vehemently for the use of the term Byzantine says it applies to the "latter eastern Roman empire". So it shouldn't be used for this city, since it's neither form the Eastern empire (which, again, did not exist yet) and especially not from its latter period.

So I shouldn't call them Romans, since that is an English translation of what they called themselves, and thus foreign to them.

But you can call them Byzantines, an English term invented in the 19th century, because... you say so? Thank you, I have been enlightened.

"Constantine split the empire" is just incorrect. There were co-emperors with regional responsibilities both before and after him. They were co-emperors of one empire. If you want a firm split, 395 (death of Theodosius) is the more common date AFAIK.

I'm not denying that gradually, over well more than a thousand years, the empire (only surviving in the East) changed character in many ways. I am denying that a unique Eastern character sprang up (and immediately applied to Egypt) the moment that Constantinople was founded.

One of the articles mentioned inscriptions in Coptic, Latin, and Greek. The elites would have spoken both Greek and Latin, I think? Greek was commonly known and used. Caesar, a Very Latin Roman, as he was being murdered in Rome, is conjectured to have spoken Greek to Brutus, another Very Latin Roman.

The notion that people are mostly monolingual, and that language is very closely tied to a cultural identity, is a modern projection (and far from universally true nowadays as well).

Proof to your modern eyes. Not to the eyes of those who also called Constantinople "New Rome", and who called themselves Romans even 1000 years later.

Not singling you out, all the articles do it, but: Strange to call this Byzantine. The coins found are from the reign of Constantius II, who at times ruled the whole empire alone (as did his successor Julian). The idea that the East, even when administered separately, was some foreign/"Byzantine"/not-really-Roman "other" was invented much later.

It seems pretty clear that they do only minimal live testing during the "open a ticket, implement something, deploy it in production, all while the customer is still on the call" cycles. So your second concern is probably not relevant in this particular setting.

Regarding the first, I think you're probably right, but then again, if there is a 15-minute base cost, it's hard to amortize that through fewer incremental runs of the compiler.

(Which isn't to say that I think they are doing the right thing.)

My thought reading this article was: Why write the system in any one language at all? And I don't just mean having some parts in one language and others in another language, I mean redundant implementations of the same parts. You can use an AI to rewrite parts of the system, and then throw away the old part... or you could just keep the old part.

That is: Have a Haskell base system. Have a Python "development" version on which you iterate at lightning speed. But also, in the background, moving at whatever pace it takes, have an agent running that imports all the Python development changes into the Haskell version. Have nightly builds of the Haskell version to reap its benefits (issues caught by the type system, more efficient native code). They must have continuous or nightly processes to fix bugs in the Python code anyway, there is no way that all the things they ship "while still on the call with the customer" are always tested on the full test suite and always 100% correct.

And it doesn't have to be Python/Haskell of course. The "development" version could be a (hypothetical?) interpreted Haskell. I have no idea if ghci would be useful for this. Neither do I know if the 15-minute Haskell build time is spent in the frontend (so an interpreter would have to pay that cost too) or in code generation or linking (which the interpreter wouldn't need to care about). Anyway, these are things I would think about before I did what the OP did.

> The type system caught real bugs.

> The model can often avoid the mistake before the compiler ever sees the code.

> The type safety we gave up hasn’t been noticeable in any concrete way yet [...]

> Type safety can be a huge advantage for LLM-generated code if the compiler is helping the agent converge quickly.

Well, good to have this question cleared up once and for all :-)

What do you mean by notation like:

    Ensure y+x ≤ INT_MAX.

Is this supposed to be a precondition? Why would I want this precondition when using unsigned arithmetic?

Naturals have no 64-bit limit.

The game in question is https://store.steampowered.com/app/4630570/RUNNING_TRAIN/

How do you break a regression? A regression is breakage. Are you one of the people who use "regression" to mean "regression test"? Did Codex learn this from you? I hate it.

Define "exists". Are you questioning whether a human typed those characters with human fingers on a keyboard? Or are you questioning whether Alec Scollon is the name on that human's government-issued ID? It's not exactly new or unusual for people to use pseudonyms.

You: >>> You want comprehensive tests at every level, far more than is reasonable for a human to build or maintain

Also you: > You have to iterate on the tests, review and validate them

Yes, "maintain" is not quite the same as "review", but the line is veeery fine. I find it really tiring to review masses of tests that an agent spews out.

Especially because I know what it has a tendency to write irrelevant/vacuous/useless tests. It's insane the amount of times I have told Codex to "write a test that reproduces the reported bug, SEE THE TEST FAIL, then implement a fix", only for it to guess an irrelevant test, not run it to see it fail, and implement a code change that has nothing to do with either the test or the actual bug.