Comment by ModernMech | Hacker News Reader

Comment by ModernMech | Hacker News Reader

Parent Full thread

> Understanding the code is NOT needed.

Programming languages are designed, not generated. What you're describing can get you an implementation for a thing, but getting to an implemented thing has never been the core issue in programming languages; indeed since the beginning of the field, implementations were nice things to have but the language itself was a different artifact. What was important were the properties of the language, which came from a careful consideration and balance of tradeoffs, and that's not something you get with "implementation of a unique programming language" because that can get you anything.

Languages are only interesting to people if the language has particular guarantees which are important for particular use cases important to the people. And we know this because before LLMs we also had a situation where anyone could write a programming language (and they did), so even before LLMS we had 1000s of low-effort implemented languages no one knew about or used. LLMs have accelerated this but it hasn't changed any of the fundamentals about how PLs are valued: the implementation does not matter, it's the language design, community, and ecosystem.

Prompt: "Generate a language that meets the requirements ModernMech is talking about."

Then I iterate on this in a QA loop. I don't need to understand. I just need to test the high level details. The input and outputs and make sure everything works. The implementation details I don't need to understand at all.

I can tell the LLM to focus on whatever use case I want adjust the language to fit whatever preference I need. Even if the inception of the idea is blurry the LLM can crystallize a starting point and the "design" of the language can arrive piecemeal in the test and iterate loop.

That won't actually work though, it'll just go off and do nonsense, I've tried that.

> I don't need to understand. I just need to test the high level details. The input and outputs and make sure everything works/

If you want it to be something maintainable, no you can't do that. If you try, you'll get parallel interfaces, half baked APIs, a bunch of special cases that are incompatible and brittle, and eventually it'll have to be refactored and good luck with that -- the agents have a special failure mode there that is quite fun (building a artifact verification cathedral and then spending all its time verifying that instead of working on code).

> Even if the inception of the idea is blurry the LLM can crystallize a starting point and the "design" of the language can arrive piecemeal in the test and iterate loop.

This is why languages are designed. This process won't produce anything but mush.

> I can tell the LLM

Okay but have you though? Have you actually tried building a language this way? How long did you maintain it for? Can we see it?

Anyway, even if what you're saying is 100% true, all that means is the bar is higher. Language implementation would mean nothing in that world, design would be everything. Which, that seems... fine? But it's not the current world IME.

>That won't actually work though, it'll just go off and do nonsense, I've tried that.

I've tried it too. It worked.

>If you want it to be something maintainable, no you can't do that. If you try, you'll get parallel interfaces, half baked APIs, a bunch of special cases that are incompatible and brittle, and eventually it'll have to be refactored and good luck with that -- the agents have a special failure mode there that is quite fun (building a artifact verification cathedral and then spending all its time verifying that instead of working on code).

You're just saying this. You haven't actually tried having an LLM write all the code and have the LLM do the maintenance. Your statement lacks any real world data and it's just wishful thinking. The answer here is actually quite complex because we have people like you who complain about it and say they "know" it won't work because they've seen it with their own eyes while people like me have seen it totally work and be totally doable as well.

So who's reality is true? Only time will tell. But I find it unlikely you've tried to have an LLM try to maintain an active project in prod. I have.

>This is why languages are designed. This process won't produce anything but mush.

Typescript is the result of years of iteration on a tiny language called javascript.

>Okay but have you though? Have you actually tried building a language this way? How long did you maintain it for? Can we see it?

Nope. haven't done it. But I have done things more complex then language design. And I can't show you because my employer owns it. But FYI language implementation follows very common patterns and this makes it very very easy for LLMS to create one. A beta of a language can be done in about a week.

The thing is, stuff like projects with a lot of low level code (such as building your own programming language) is not THAT good at being AI generated. Personally, I use the like manual version of claude where you have to like approve everything it does because when I was testing it out myself (for your "real world" data), the LLM would write code in a HORRIBLE way that would be redicioulously difficult for a human to optimize in the future because, there is a limit to how fast the code that an LLM generates can be. often times what ive seen is that it just does things the "quickest" way or the most "technically correct" ways (i.e functional programming or using a lot of pre built APIs which introduce overhead etc) which arent the best ways or the most optimal ways.

An experienced human can write code better than an LLM for low level tasks almost 100% of the time, the only problem is its REALLY slow for a singular person to write code when compared to like an LLM running at 600 tokens per second or something. The best combination of the two is (in my opinion) running claude on manual mode cus if it does something really bad you can either fix it yourself or reprompt it, but there is a LOT of active things you have to be doing.

>Nope. haven't done it. But I have done things more complex then language design. And I can't show you because my employer owns it. But FYI language implementation follows very common patterns and this makes it very very easy for LLMS to create one. A beta of a language can be done in about a week.

its not about the fact that the thing you are doing is "more complex" (in my humble opinion the only thing more complex than language design is building an OS) but its more of the fact that if the LLM writes one line of code that is not like optimal for performence, your language will like leak memory or it will run really slowly, or something will happen suboptimally. you cant just leave it autonomously for large periods of time (I have tried and regreted this) but you can still use it, as long as you read over and esnure that the code is optimal. if you dont do that we are just moving backwards in time and the code you write is very unmaintanable, slow, leaks memory or is volatile, and impossible to scale up or down.

Reply on news.ycombinator.com