Hacker News Reader

Comment by ohmygoodniche | Hacker News Reader

The amount of cognitive dissonance here is interesting.

I compiled c to asm. Title says the llm did this. it works! But it's broken. It generated a bunch of other files! But I only need one. It couldn't target z80 so I was a human in the loop. You have to trust it and understand how the Black box works to get n factor gains. But no one knows how these tools actually work and general advice is NOT to trust LLM outputs and the author didn't trust them either... And even the final result has the incorrect tax rates...

I'm not denying LLMs can sort of rewrite small chunks of code in other languages, add comments to code, etc. but the way people talk about them is so snake oily.

Going by any of the major bullet points I would say that the title is wrong, and misleading at best.

View on HN

I've tried to get LLMs rewrite a bunch of stuff in Rust, from different languages (JavaScript, Python, C++, C) and I can definitely relate: LLMs cannot be trusted rewriting anything significant without a lot of supervision (realistically, the only thing I've gained was not to have to type boring boilerplate, but that's pretty much it).

And before you say “oh but it's because Rust is too hard”, SOTA LLMs don't have much problem writing Rust code nowadays, and I suspect Rust is actually a better candidate than most language as a target, because the compiler catch so many things and the errors are very explicit, which helps the LLM a lot when doing multi-turn rewriting sessions.

> Going by any of the major bullet points I would say that the title is wrong, and misleading at best.

And it wouldn't be their first time.

Check https://news.ycombinator.com/item?id=43217357 by the same author. That post got flagged for being misleading too.

Read https://news.ycombinator.com/item?id=43220639 for same type of criticism.

The OP seems to be on an LLM spree. They ask LLM to produce code. The code is invariable always broken thanks to hallucinations but they go ahead and post it on HN anyway with a misleading title.

https://ghuntley.com/about/

Ah well maybe the blogs are written by an LLM to try to gain "agency" or test how compelling their poopy code is against people on the internet for free... I cannot imagine what would motivate an actual person to be that publicly wrong and frankly hostile about it while maintaining any sense of reputation...

You're missing the point.

The point isn't and never has been that it's a flawless tool. The point is that you can work the tool and get a working POC of something you'd never attempt to do before in literally a couple hours.

It isn't a compiler, it isn't a logician, it's a lossily-compressed image of the whole internet with English as a query language. Use it within the operational envelope, which is what the author did, with some interesting results and possibly pointing to a large implications on vibe-coding solutions you'd previously pay for.

For this piece of code, one can rewrite it correctly and much more performant in much less time. If the tool only drags you down, you'd better drop it.

The estimated run time of this code provided by another poster would have compelled me to seek alternatives as quickly as possible that's for sure...

Just a note that - 'one can' does not mean everyone can. And this can be applied between any two languages or tasks. An AI will simply have better broad knowledge than practically anyone at a task they are unfamiliar with so they can massively reduce the friction of getting started.

If you cannot exam the result, you better not touch anything LLM generated.

Again, missing the point.

You can. I can't. I don't want to. I don't care if it's slow, I can easily tell if it's correct and I can make the LLM fix incorrectness (in this simple case, anyway).

The point is this project probably wouldn't have happened without an LLM.

This "project" is a toy application that exists for no other reason than vibe coding evangelism.

> You can. I can't.

> I can easily tell if it's correct and I can make the LLM fix incorrectness (in this simple case, anyway).

Pick one.

I'm pretty sure black box testing is not a dark art around here.

Getting the right answer with the wrong process is still a failure.

So even if the result happens to be correct for the example you gave it, this process may not be able to produce correct results for other cases. Or even this case again.

And if you can't understand the process, then you won't know when that happens.

Now, if you're just trying to sell the result. I guess, no harm, no foul. When the process breaks, you'll be the one affected. That's fair play.

But if you're trying to sell the process, you should be able to verify the process. Because otherwise you are selling a broken tool.

Are you just running code you don't understand to see what happens?

I'm reminded of the infamous hn comment when Dropbox was announced

> 1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.

https://news.ycombinator.com/item?id=9224

This project actually didn't happen even with the LLM.

The sounds like exactly the point OP is making. The way LLMs are spoken about implies much more than actually demonstrated.

> The point isn't and never has been that it's a flawless tool

It's a contradiction with all the managers and vibe-coding developers that have been saying for months that it can replace 90% of a development team.

I'm going to go against the grain and say that AI could feasibly replace a very large percentage of most development teams, even today.

Not because the AI itself could do all of the coding with no developers at all in the room, but the developers who do know how to use it effectively could output so much more than those who don't that they would more than make up for their lost productivity.

Lots of companies have not yet fully embraced AI, and their developers are actually held back by not having access to it as a tool. But as someone who was recently given full access to sophisticated AI tooling, it makes a massive difference in my productivity.

Lots of people don't like to hear this, but if you are not using AI today in some capacity, or your company is lagging in its adoption your career is at risk, especially if you are still relatively young. This said as somebody who originally was a massive AI skeptic, but decided to give it a shot.

Yes, you still need to know how to code. That is not going away. But there will come a time when you yourself write an order of magnitude less code than you do today because you will become more of a reviewer than a developer yourself. Software development as a role will still exist, because in essence our job is to solve problems and build software, it just happens to be we write a lot of code to do that nowadays. But we will reach a breaking point where we don't write much of the code ourselves at all, maybe just some edits, and we review orders of magnitude than we do today.

The fact is that for a lot of projects, you don't write a lot of code. And for the occasion that you do, the issues lies mainly in integrations and requirements/design cycle. I learned Vim and Emacs, not to write code faster, but to edit it faster. I've never been in a position where I said: I wish I could write more code.

In fact, most of my coding happens in an unfocused state. It's when I'm reading code (when there's a gap in the docs), learning a new system (libraries, platform, language), or designing a system (architecture, integration,...) that I give my full attention. The actual writing is mostly Edit/(Compile|Lint|Test)/Fix cycle that actually goes pretty fast in term of iteration and don't use that much mental energy.

> Lots of people don't like to hear this, but if you are not using AI today in some capacity, or your company is lagging in its adoption your career is at risk, especially if you are still relatively young. This said as somebody who originally was a massive AI skeptic, but decided to give it a shot.

spot on - key reason behind authoring https://ghuntley.com/ngmi/

> Yes, you still need to know how to code. That is not going away. But there will come a time when you yourself write an order of magnitude less code than you do today because you will become more of a reviewer than a developer yourself. Software development as a role will still exist, because in essence our job is to solve problems and build software, it just happens to be we write a lot of code to do that nowadays. But we will reach a breaking point where we don't write much of the code ourselves at all, maybe just some edits, and we review orders of magnitude than we do today.

spot on - key reason behind authoring this https://ghuntley.com/multi-boxing/ - I see a future where exactly that. SWE's spend more time reviewing code than artisanally crafting it by hand. Instead of the IDE we use today being the tool, it will become something only used exceptional circumstances. Instead, our primary tool will be a code review tool (that doesn't suck) that drives agents.

There are areas (IME - React internal tooling frontends) in which the only missing piece is the 'review UI' and it isn't even that important - vibe coding is good enough. My point is I've been living in the future you speak of for the past couple months and I love it.

Give it a try. You'll be surprised at what can be achieved when the LLM is driven via /specs (business requirements) + /stdlib (steering LLM technical outcomes). The end result, when driven by a good eval loop (property-based tests + compiler that provides soundness such as Haskell or Rust), is code outputted at brrrrrrrr speeds which is high quality.

Having seen and helped fix colleague and stranger generated LLM generated rust and typescript code I would rather not rewrite code all day to make it hold water. Doing that for Haskell would probably give me an aneurysm.

Every test I've ever tried with an LLM to get it to generate code has produced a complete unworkable mess. I have no idea what people are generating with them, but its always far less work to write something that I understand, rather than spend the time trying to fix up a complete disaster that barely even begins to touch the problem I asked it to solve

I'm not saying this is you but I have to ask as someone who has had success using LLM's but originally had this same mentality - what is the most recent model and tool you used to try to generate workable code?

If you tried it even just 3-6 months ago, in just this small amount of time tooling has had such a massive improvement that maybe you haven't tried it recently. I had your same experience when I tried before, but I have readily been able to get LLM's generate thousands of lines of actually useful and readable code for my job.

I tried generating code before and dismissed it because I had similarly bad experiences. But having generated myself now entire apps where I barely wrote any code that were actually usable and productive, including internal tooling, personal apps, and code that is client facing in production today, I don't think this really applies anymore. LLM's are more than capable of producing lots of high quality code given the right tools.

You’re digging your own hole by using “brrrrrrr speeds” as a marketing term. It doesn’t help your overall argument.

Reply on news.ycombinator.com