Hacker News Reader

Comment by bitbasher | Hacker News Reader

29 pointsbitbasher27 comments

I'd consider myself a very experienced (~30 years), but mediocre dev, and this AI thing has completely transformed my capabilities as a software developer. People compare AI to a "smart junior" or something like it. To me, it's more the mentor I never had. When I have AI review code that I wrote, I will point out things I would either have never thought of, or would have taken me weeks or months of going back and forth to figure out. There are lots of things in software development that I hate doing, such as CSS/HTML, AI is now filling the gap that used to be an obstacle for me. With AI this now has become fun as it feels like I am not alone working on this thing. What it produces, I can understand and I review its work as well as vica-verca. I mostly use it in assistant mode. I do not have an army of agents running (yet).

I don't know how people steer multiple agent sessions in parallel. The constant cognitive switching is exhausting and you miss moments unless you are cli jumping. It is like being an operator.

It would be exhausting, if it were always necessary. If you spend the time up-front to properly define goals and requirements in a machine+LLM verifiable way, then the agents can manage it in isolation or with minimal oversight, and present a working result that meets those criteria.

BUT: How many people do you know who can achieve sufficient clarity up front? It is a skill (or set of skills) that needs developing. It can also mean the difference between spending $20 in tokens versus $2000, and/or throwing away the result and starting from scratch again (you don't really want to touch an AI generated codebase with fundamental design flaws if you value your time and sanity)

In the meantime, deliberate checkpoints for human review are still a good idea.

My theory: behind every "10x AI coder" is a long trail of expensive failures that never made the light of day, but which they are learning from. The early adopters will therefore have a competitive advantage.

I have been thinking about this and I think it might work well for established code bases that already exhibit specific patterns that are used over and over again. E.g. a CRUD app with a OpenAPI spec, a controller layer, a service layer, a repository layer, or even good old MVC for that matter. With these established patterns it wouldn't be unthinkable to provide an orchestrator agent a set of requirements that are detailed enough for a group of agents to implement, each agent focusing on a different type of task including an Q/A agent. I'd imagine you end up with one or more PRs for you to review and squash & merge when you're happy with it.

Edit: What I mean is that these agents would work autonomously until their task is accomplished. They can ask for clarification if they require it.

Only use chatting to get answers for various things, often cross referenced with Google results, dedicated forums, and reddit. For programming/software work I use it to try to find problems with my architectural/design decisions, find new libraries, and best practices. I do not use it for code gen, instead I leverage high level deterministic (non ai) tools to do more with less code.

It's not going to tell us much because we don't know the standard of code quality that respondents hold themselves to.

True, but easier to see consensus than reading through a thread that is filled with a variety of posts in terms of usefulness.

Yes-- this is what I wanted. I wanted to get an idea of where most people fall.

This has to be the first poll I have ever seen on this website

They use to be more common, but polls have low odds of being productive.

I hope this thread/poll sparks productive and effective discussion on the subject.

Yes, just also be sure to spend some time writing "by hand".

I agree with this, but I’m also curious: what would have to change before that advice is as sound as “write a little bit of assembly by hand” or the even more ridiculous “just write the raw bytes for the program in a hex editor?”

Well, I'm currently applying for jobs, so being able to write code without AI is actually important.

More generally, I want analytic reasoning and problem solving skills. Assembly, C, and Python all still have me writing an understanding algorithms, while prompting (mostly) does nt. (Actually, more so with C and Python than Assembly because it abstracted away an appropriate amount of stuff, much the same as how I can do better math with pen and paper than in my head.)

It's possible that at some point it will make sense to switch to a different analytic reasoning practice regime, but for now, programming is a really relevant one for me, and one I enjoy.

A compiler is a reliable layer of abstraction using documented structured languages. For me it would need to be that.

Even with LLMs, we need a way to translate between the imprecise plain English description of a program and the completely-unambiguous level of code. You need the ability to see when the LLM has resolved ambiguities in the wrong direction and steer it back. If you can't speak code, that's going to be a very error-prone process.

When I can look at a prompt and predict what the code it outputs will look like to some high degree of accuracy.

I mostly don’t think that is possible though because there’s too much ambiguity in natural language. So the answer is probably when AI is close enough to AGI that I can treat it like an actual trusted senior engineer that I’m delegating to.

Can you look at code today and predict what assembly a compiler will output to some high degree of accuracy? Do you avoid certain classes of compiler optimization so you can more accurately predict compiler output? I recall a time where many compilers would remove a bzero() operation in situations where you’re trying to zero out a buffer that had sensitive data in it - it’s why we have APIs like https://github.com/MicrosoftDocs/win32/blob/docs/desktop-src.... I ran into a huge performance regression because I didn’t have all the edges of named return value optimization in mind when I refactored some code.

There’s ambiguity in the x86 specification, such that you can execute a single instruction and get different results in intel vs amd. See the rcpss instruction, for example.

I get that LLMs are categorically different, and they’re absolutely not as reliable as compilers are, but compilers are also not as reliable as compilers seem. And even less predictable IMO.

> Can you look at code today and predict what assembly a compiler will output to some high degree of accuracy?

Yeah a fair amount is the time. And when I can’t, I can predict what an unoptimized version of the assembly will look like.

And I know that the optimized assembly has a very very high likelihood of being semantically identical. And I know enough of the edge cases where the differences matter to know when I need to actually verify what’s coming out of the compiler.

Prompt instability (not even worrying about non-determinism ) ensures that asking for the same exact thing in very slightly different ways or with slightly different contexts will give you wildly different outputs that are not even close to semantically identical.

I don't think it's wise to use it for anything. Even when chatting with an LLM you would have to check everything yourself which nobody would truly do. The generated code can't be really trusted (and nobody will review everything, quite the opposite). It can also have copyright issues.

People are allergic to articles and documentation generated/processed by LLM.

You're switching from an active role to a passive one, meaning your skill will suffer over the time. There is a huge difference between doing the things and thinking you know what it's doing. It's harder to review bad generated code because how polished it looks, compared to code made by humans where the difference is much more obvious.

Code assistants seem to work great when dealing with boilerplate, but wouldn't be better to get rid of the need for the boilerplate in the first place?

There is a category missing between "as much as you can" and "chatting".

Yes, as a better autocomplete.

I was made redundant (from a web dev job) a couple of years ago, and have been looking for a new job. But the thought of coding with an LLM gives me the heebie jeebies. The very idea makes my skin crawl. I think I need to find a new industry. I don't yet know what.

For me, I used it find libraries to solve a problem that I didn't know about. Or to debug confusing error messages when library/language docs are insufficient.

Are you asking what I do or are you asking for advice on what you should do?

Yes

Sounds like neither. More like throw a dice and press the up button.

My flow is the AI writes most of the code, I closely review, question and tweak everything that comes out. My commits are about the same size as they were. Don't vibe code or one shot features.

Reply on news.ycombinator.com