Hacker News Reader

Comment by iandanforth | Hacker News Reader

Wut? I pilot LLMs all day but there's no way in hell I'd agree to be at the helm of a finance product. That first pillar is still there. Maybe the author isn't aware of the impact they have, but I know, with the evidence of reverted PRs, that when I step outside my area of deep knowledge I can no longer call BS on the agents. Our most capable agent, with access to the same kind of distributed systems the author talks about, is regularly wrong, frequently myopic, and just outright dumb constantly. It's the expertise of engineers on the team that push it back on track.

View on HN

Posting this under a burner so I don't dox myself: I work in FinTech on a regulated product. We have access to Mythos. Mythos identified part of our codebase that it confidently asserted was not complaint with a particular regulation and we were at grave risk by allowing it to operate the way it was.

Except this was not the case, it had of course hallucinated what the regulation actually required (I know this because the code in question had already been reviewed by human counsel). This is (supposedly) the most bleeding-edge model available.

We use a lot of genAI to help us write code, but there is no way in the mid-term we could ever rely on these tools to actually build compliant financial products. We'd have to be totally mad. Yes, lots of Fintech companies are using these agents to accelerate, but anyone who's using them to actually ship product without a human actually digging into it is opening themselves up to a world of risk.

I have worked on highly regulated areas in finance (risk). Compliance is a highly creative art, often requiring lots of out-of-the-box thinking and non-obvious solutions. The people I found worst at this were IT. They tend to over-interpret regulation, and super-restrict beyond what is needed for actual de-facto compliance.

My guess is the model makes the same mistakes as the programmers: taking 'rules' literally, unaware of sectoral joint understanding, validated interpretations and habits. (btw. this is often on the non-tech side also a difference between regulatory and legal. The former are much more result oriented while the latter are primarily risk averse.

Ha. I've worked in a fairly strongly regulated sector (energy, in the Netherlands), where I collaborated closely with our head of compliance, and she heavily over-interpreted the regulations while I often tried to find more pragmatic solutions.

I think adherence to regulation and compliance is nothing to do with whether you're a SWE, a risk officer, or C-level, and everything to do with your own principles, ethics, professional attitude, and pragmatism.

I've found two things to matter:

1. experience, i.e. knowing why and how a rule matters (in general, but also to auditors)

2. willingness to think

If these aren't present, you get overly restrictive compliance that at the same time accomplishes nothing.

I'd also add a willingness to follow the (perceived) intent of the rules as opposed to gaming the letter of the rule. An experienced folk might say, "yeah it won't matter legally", and continue with "but it will matter to me because this rule is in place for this-and-that reason".

scruples are also often a surface of friction that get in the way of business objectives.

It's not even limited to SWE - there are so many regulations in almost every trade (which sometimes contradict each other) that it often eventually boils down to your experience and your decision which rule to follow and which to 'bend' (for example in "traditional" architecture there are so many rules that the building authority itself has to compromise on following the law by the book).

> IT. They tend to over-interpret regulation, and super-restrict beyond what is needed for actual de-facto compliance.

IME this is less the fault of IT and more so bad auditors that won't consider, or just don't understand, what compensating controls are. If it doesn't meet their little checklist exactly, they fail the audit.

> IT. They tend to over-interpret regulation, and super-restrict beyond what is needed for actual de-facto compliance.

This is such a nonsensical claim. If a company is asking someone from IT to read the regulations and implement them, then obviously you’re going to get something that conforms to the written specification they were provided.

But a company that does that is basically delegating both compliance and legal functions to IT. No sane company does that.

> But a company that does that is basically delegating both compliance and legal functions to IT. No sane company does that.

I was a Software Dev in a small (but fully regulated and licensed) stock exchange. We used to have guidance from legal experts, market experts, and traders, but in the last project I worked on, they just dumped 300 pages of laws and regulations on my desk and asked me what needed to be done. Why? Because the experts we used to have were either fired or left. Along with any product managers. I guess company leadership thought they were no longer needed.

Insane is right. I told them that this is not how it is supposed to work. I can't tell them what needs to be done. I am not a legal expert who can just interpret these regulations.

I was forced out of the company after that, but honestly, no one would want to work in such an environment anyway.

> But a company that does that is basically delegating both compliance and legal functions to IT

This actually happens scarily often, especially in smaller companies. No F500 is doing this, but there are tons of "mid market" sized non-tech companies (think 80 to 150 employees in size) that basically rely on the IT department of 1 or 2 people, or an MSSP for pretty much everything. No legal team, maybe an attorney they consult with once or twice a year if you're lucky.

That's actually pretty important, if you're an eng doing compliance work and you don't have legal counsel working side by wide with you you might be putting yourself up for legal troubles down the line. I'm glad I can always rely on legal do do their job here when doing this kind of work, I wouldn't want to do work like this just out of my ass.

perfect IT response

regulation are written ambiguously and the specifications do not match the industry

I have even seen regulators refuse to specific legislated laws because "thats not what the government meant", giving a company the choice of following the law and being fined, or breaking the law to please the regulatory agency

Regulators don’t want to provide arbitrarily detail into their interpretation or likely judgements on issues that may come up for many reasons, good and bad.

One good one is that providing concrete razors for compliant and non-compliant behavior accelerates the gaming of the rules.

> compensating controls

How to say you deal with PCI compliance without saying you steal with PCi compliance.

Compensating controls also come up in the context of BSA/AML.

Also in IEC 62443

It's cause IT never has to live with the consequences of their decisions. Who cares if the other department keeps bleeding talent because you twisted the knobs so hard no one wants to work in your system?

Sounds like communication between departments sucks. If IT develops for them, you’d expect there to be a feedback loop?

Yes. Exactly. This is not a reflection of where I am now in any way shape or form. Just my observation of previous places I've worked.

Why would they care? They take the blame when it gets hacked, but don't really get any upside for bending the rules to make people work easier. CYA rule-following is just to be expected.

> he people I found worst at this were IT.

My experience as IT in modern banks was the opposite. The legal department were absolute assholes when it came to software features. And I'm talking absolutely 100% ok features, like paying your bills from the banking application.

The least fun, trigger happy, cover their buts people I've ever seen.

Like all they could ever say was NO. I guess they were heavily incentivized to just say NO to everything.

I remember working with a corporate customer whose in house architecture team was so difficult to work with that I joked that they could be replaced by a rule in their email system that simply replied "No" as the message if I ever emailed them.

That was my experience too. Legal wanted things locked down hard. IT was more than happy to make things easier for users as long as legal wasn't getting in the way. Usually meant the systems were simpler too if we had fewer rules and controls to enforce.

Auditing is assumption of blame or responsibility to another party.

They are incentivized to strike the best balance of certifying (who wants to go to a place that never certifies) and validity (rubber stamp mills reflect the blame).

Yes, it is meant to be adversarial, to a point.

I don’t think you’re going to find a consensus on this because it really just comes down to the quality of the employees in each discipline. Actions speak louder than words. I’ve seen the IT people GP is describing. I’ve also seen yours. In GP’s scenario, they often even mean well but are very overwhelmed because they’ve come to exist in a space where _everything is IT_ because no one else is remotely qualified to fill the specialty gap. When I found myself in your scenario, the opposite was true and it completely matched what you described.

I once worked on a data compliance job, and the auditor would fail everything he possibly could. He was there for data destruction compliance, but like many such people, he came from an engineering background. He would complain about everything. Gaps between pallets. OHS. Whatever he could to justify his decisions. He never found a bit on a disk out of place and he still made our lives hell. Failure for the floor didnt feel solid enough. Failure because he didnt feel comfortable in a warehouse environment. And when the management had had enough and decided to refuse him entry and ask for someone else, we had to hold ourselves to an even higher standard to compensate.

Later I worked in a role, attempting to achieve PCI compliance. The Auditor was a really nice guy, but there was always a short list of 10 things that he wasn't quite happy with. We kept increasing the scope of compliance to keep up with him. Everyone talked about him (Semi famous local celebrity security consultant/researcher/lecturer) and claimed that if we just stuck it out we would be super duper compliant and basically unassailable. Except that it never ended. Went 12 months with the guy. Then they just stopped paying his bills and brought in another auditing firm. Compliant immediately. You never know in a situation like that whether we were actually compliant or if there was graft. But we got there. Knowing that organisation I lean towards graft. They then failed their first audit after achieving compliance.

I have done a few PCI compliance operations since. And what I have found that you cant control for the auditor, so what good IT management does, is make every single requirement completely unassailable. If you cant write a very obvious compensating control in 5 sentences, then you just move heaven and earth to comply with the letter of the requirement (even if the project to become compliant, is itself a compensating control for a while). If you get an over achieving auditor, you wont spend 200 billable hours arguing about compensating controls. If you have a shit auditor, you know you are compliant even if they aren't being as thorough as they could possibly be. Its the only ethical way to navigate the situation.

Who gets in trouble if it turns out you are actually held to the literal rule?

Contrary to what you indicate rules are not declared in a vacuum, for people to read and then algorithmically 'implement'. There are many ways to interpret regulation, and there will be both accompanying clarifications, as well as compliance departments negotiating with regulators on what is an acceptable and sufficient compliance action. Then there furthermore is a risk that will be calculated vs the cost and opportunity costs etc.

As an enterprise architect, these are all part of the meetings you have with compliance when you are working on major projects. I have had the privilege of working with some excellent compliance officers, and they are the opposite of the nay-saying caricature that is often painted of them. I found these people to be extremely creative and helpful, working together towards solutions rather than stalling or nixing viable progress.

I also work in finance and my recent experience with regulators is really discouraging. DOGE wiped out a large amount of the regulators in government. It seems like most of the regulators remaining are the inexperienced and low tenure. Within the past few months we've attempted to roll out new financial products. When we attempt to send our proposal to them, they can't even tell us who we're supposed to send it to.

It doesn't feel like we're living in the same world of regulation that existed prior to DOGE.

So the DOGE geniuses failed to remove the regulations they allegedly thought were hampering legitimate businesses, while removing the people capable of verifying whether or not your business is in compliance.

What a win!

    > I also work in finance... DOGE wiped out a large amount of the regulators in government.

I found an insanely detailed Wiki page about all of the gov't divisions affected by DOGE: https://en.wikipedia.org/wiki/US_federal_agencies_targeted_b...

However, I don't see anything about finance there. I'm confused by your comment. Can you provide more specifics?

The point was about who is on the hook and why they might be less permissive.

I'm not implying anything else. I used your own "literal" wording to refer to the "more strict than yours" interpretation.

I suppose I should have used scare quotes around "literal".

'The company' would be on the hook. Inside, it might be the compliance team that signed off on the solution, but it usually is not the sort of blame game at that point. I'm not saying these scapegoat trails do not exist, but they are far less common than you would imagine if you only read about them in the press.

Company politics, feudal wars, fiefdom protections, backstabbing and outright sabotaging, now there's a daily occurrence and many minions are cannon fodder in those skirmishes, but they usually stay clear of regulatory issues minefields.

> There are many ways to interpret regulation,

Then the rules should enumerate all the ways. From your posts, you come across as if programmers don't know what they are doing which is insulting to those who work in mission critical industries like aviation where a programmer could be criminally charged if he/she didn't implement the specs STRICTLY.

> Then the rules should enumerate all the ways

It's nice to want things, but rules are much squishier in real life. There's rarely any truly bright line.

"you come across as if programmers don't know what they are doing"

Is neither what I said nor believe.

That's why you work with your Legal/Compliance Team to make sure you stay in line. They can explain when a rule applies and when it doesn't. This needs the engineering side to be able to explain what's happening, and translate it into the business process as closely as possible, and the legal side to be able to apply the law to the case.

If you think rules are literal than you aren’t aware how the world works.

There’s a reason it’s called “judgement”

In your world, do subordinates ever get scapegoated for bending the rules at a boss's behest?

...And that judgement could take them literally. So what is your point?

My point was simply that it's easy to scoff at someone else being careful if it's their neck and not yours.

They could but they don't. That's pretty much the whole job. You can also appeal decisions to a more reasonable party if you draw RobotJudge3000 for your trial

> mistakes as the programmers: taking 'rules' literally

Isnt it how we make stable, deterministic and predictable system? How do you want to create one with ambiguous rules?

It’s not a highly creative art at all, genuinely don’t get this weird compliance glazing here.

The dynamic of agent codes human reviews does seem like the only sane one for the foreseeable future. Even Anthropic themselves still fall back to this.

The problem is that sucks, even if all software engineers keep their jobs and salaries, the floor is still pulled out from under us. Imagine if a surgeons job was to supervise robot surgeons from a remote computer, or a woodworker just signs off on work before the machines do all the cutting and assembly. Sure they still have important jobs in their field but the soul & humanity of their skill is gone.

I never found there to be much soul and humanity in the job to begin with. Coding personal projects has soul, but for me at least the demands of high-velocity sprint-based software development to match business needs removed most of the soul and humanity long before AI got good at coding. And I mean, I totally understand why it has to be like that. In most businesses, you do better by shipping decent software fast than by shipping great software slowly. I don't have a problem with that in principle. But it does mean that for me, the software development side of things has never had much soul and humanity to begin with. It was just being a glorified assembly line worker, with the sprints being the assembly line. Of course, others may have had very different experiences, but that has been mine.

For me, AIs have actually made the job more soulful, not less. For one thing, it lets me use the part of my mind that is good at human language, not just the part of my mind that is good at software. This makes the job feel a bit less one-dimensional in terms of what parts of me are engaged while doing it. For another, I find it liberating to no longer have to think much about boilerplate code or to spend time roaming around the Internet looking up documentation of various language syntax and API details, the vast majority of which are arbitrary rather than being based on any kind of mathematical beauty. For me it makes the job more soulful that I can think of the job on a higher level instead of having to spend effort on arbitrary and tedious details.

Of course there is still the question of "will the job even exist in a few years, at least for more than a relatively small number of people?". But that's a separate question. For now at least, I am finding that for me AIs have brought a lot more soul and humanity to the job than it ever had before.

That's an interesting perspective. It's hard for me to relate to it because I haven't worked in a job where I just have to ship code 'for work' in so long. Being a more or less one-man software company, all my work projects, but especially our products, feel like personal projects.

However, if I were just having to do things for the man, I might have a rather different take on all this.

Yup, I can definitely imagine that it's different if you're working directly for customers and have the freedom to do things however you want to do them as long as you still make a living.

The flip side is that if you have that creative control, then LLMs have _definitely_ sucked the joy out of programming, and in the worst way.

I don't think that is true. If you have the creative control, and LLMs suck the joy out of programming, then you and I have very different ideas about what that joy was in the first place. I enjoy programming both on a very high and a very low level, and both are more fun with LLMs. On the low-level, you can use that to create the building blocks that the LLM then just has to combine. And on the high-level, you can use that to steer the design in a way the LLM would never be able to, but with the help of the LLM you can connect that high-level design much faster to the low-level building blocks.

Maybe try using them differently (I tend to use them like static analyzers I can yell at/argue with, and honestly less straining than trying to parse a Coverity report), or just avoid them. Mental health is more important than 20% gain or loss (depending on which study supports your prejudices) in productivity.

You're probably 1 in 10 000 programmers. Most programmers are just regular employees, the vast majority in non tech companies.

Yeah. It's easy to forget that sometimes.

"Soul and humanity" is doing a lot of work here.

Does the woodworker who shape using a handsaw use less "soul" than the one who uses a machine?

Does the musician who use a DAW and VSTs instead of analogue tape recorders create music with less "soul"?

Does the painter who buys acryllic paint instead of synthesizing their own dye from plants use less "soul"?

As technological innovation progresses, the barrier to creation falls. The process of creating something is not to be conflated with the final piece of art itself.

Your analogies are flawed. DAWs and skill saws generate nothing. They take skill to operate, and a novice cannot use these tools at all unless they know the craft.

Compare to this to prompting an LLM: “Generate a third person where game with a view from above where you can steal cars, shoot at people, run from the police, etc.” Anybody with access to the tool can do this, and the results are just another uninspiring GTA clone that you would imagine.

The latter is more like a carpenter ordering their “work” from alibaba then it is like using a skill saw.

> Does the musician who use a DAW and VSTs instead of analogue tape recorders create music with less "soul"?

https://www.youtube.com/watch?v=gkqNWNLKpZg

I'm critical of many "AI" developments but I can't and don't want to argue with this. I say we still need to struggle for humanity and we do need to save our souls, but that "it's a machine" is not where the battlefield is.

Does the carpenter who used to build custom fit cabinets with hand and power tools put in the same creativity when he just carries around a scanner, scans the area, the customers use software to select the layout, approve the work, then the CNC cuts out the wood, then all that's left is to put the screws in the holes and go home.

This isn't like the step from hand saws to power saws, and it's disingenuous to pretend like it is. This is what the startup machine has been doing to every industry... finding... "inefficiencies" and "optimizing" them.

Yes, hand made goods come with a story and a memory of the artesan who created them and those things have value.

It's not the item's soul that's at stake when you stop recognizing that, it's your own.

Hmm, but AI isn't just a really-good tool, it's also doing creative work too. As an aspiring composer, music generators are in my opinion really quite good, and often matches what composers and song writers can do. So if you ask me does a person who creates music with gen AI miss out on the soul of creating music? In my opinion at least, for the most part, yes.

To write a piece of music, you're working at so many different levels, the analytical, emotional, and structural as well as drawing on years of training and experience. When a person with little (or even just a moderate amount of) music training generates a piece of music in a couple hours, are they actually a composer? I personally would say no. I mean it does take a good ear (which is important) to use a music generator well, but still, would say they are more of an editor or an evaluator or a practical critic of music instead of a composer.

Yeah, at what point in a discipline does the increasing skill of AI overtake the need for our contributions from the working being done? In music, it's happened already. Looks like it's happening in coding too.

Not _my_ opinion, but I just wanted to share that many people (in the Midwest) do believe that anything synthetic that it not readily made from simple materials has "less soul". It's a sorta test of "if I dropped you off in the jungle, can you still produce works of soul? Or are you just another cog in the machine.".

Except it's not just a tool.

It's when a woodworker, musician or painter completely outsources their work and just marks what's wrong, sending those parts back. Yes, the final art piece might be the same, but the artist definitely uses less of their "soul".

> The dynamic of agent codes human reviews does seem like the only sane one for the foreseeable future. Even Anthropic themselves still fall back to this.

Do they? I saw some crazy stat from the guy who built claude code that he was pushing hundreds of PRs a day. There's no way you can human review that much code. It's probably closer to heavily AI assisted review and planning.

I don't know if I agree that's the only sane workflow; the problem is, I am way less invested doing code reviews of agents than I am reviewing code by human colleagues.

I would love to be able to say I pay the same amount of attention and am just as diligent and communicate as clearly with an agent, but it wouldn't be honest: I scan agent PRs for obvious mistakes or misinterpretation of what they've implemented.

With human colleagues I usually know them and their style, their way of working, so have a better idea what to look for. You also have a genuine return on providing feedback that helps coworkers learn and improve, whereas with agents, all the feedback you write is gone when the thing gets merged (unless your org has some kind of shared memory for its agents).

I don't have the answer for what the future looks like, but I suspect agent-type-1 reviews agent-type-2 is actually where we'll end up.

I think there is a big difference between a surgeon, who is performing a specific task with a clear outcome, to a woodworker, who might produce a unique piece of art or a functional chair. I think the surgeon-type tasks will be replaced eventually. More interesting are the woodworker types, which has some similarities to SWEs.

When industrialization hit, we definitely lost a ton of craftsmanship and craftsman, but a standard Ikea chair is less likely to wobble than the average chair at a much better price (for a random example). Yes, we traded artistry for convenience, but what we really did was bifurcate our needs between "some place stable to sit" from "a beautiful chair for my home". Most people wanted the former more than the latter, and the same applies to software.

If we split the roles into buckets, many woodworkers disappeared, some became artisans, some became designers for industrially-produced products, and some catered to Luddites for a long transitional period. Despite Anthropic's claims, SWEs won't disappear in a year but over a generation or two, no matter how good LLMs become.

Obviously software is much more complicated and integrated into other elements of business, which in a way makes it more vulnerable to AI taking over and in another way will be at the mercy of larger shifts to how businesses organize human roles and responsibilities. What we call "taste" comes down to "intent" - what the hell does a company do? What should it be doing and how should it operate? These will be the only questions that matter and the one thing LLMs can't replace since they will always choose the most default path. So I think human's roles will be to inject intent/taste at different levels of abstraction throughout an organization.

Im not sure your assumption holds at all. If anything the ikea chair could argued to be a very efficient use of resources producing a minimally useful chair. But why is it less likely to wobble ?

In addition the incentives are misaligned - the "artisan" made chair (in the past) wasn't likely made for aesthetic reasons - it was made to last long term and function. And if it wobbled or had any problems the original woodworker was probably around to fix it.

And yet, I ended up throwing the IKEA coffee table out, and instead making one myself.

I had no prior experience with woodworking, and while it's a little bit wobbly, it's much more robust and it's the exact shape and size I need. It cost the same and it'll last much longer.

And there's emotions and a story attached.

After a couple of years of this their expertise will be gone too and then nobody is qualified to supervise the clankers.

Only sane dynamic is human writes code, human reviews code, ai also reviews code.

I don't believe there is any point in having ai generate code

> but the soul & humanity of their skill is gone.

If robots make life saving surgery cheaper so that many people can afford it, isnt that a good thing?

100%. Unfortunately those not in the depths of mission critical systems or regulated products will continue to believe that producing tons of code quickly using LLMs without humans in these systems is acceptable.

Here's an example of what we will continue to see with folks fully immersed in gen AI psychosis:

"The creator of claude code said that he no longer writes code for about 6 months and now has Claude doing all his work now. He also said recently that he no longer prompts Claude and now has it running in loops and it is self-improving itself and performing better than a human!"

If the code produced by the LLM is perfect, the LLM takes the credit. But when a disaster happens, you cannot blame the LLM and it then falls on the human who did it.

I don't think SWEs heavily vibe-coding with LLMs realize the risk in not understanding what the code the LLM being produced is doing even after generating tests (lol). We will see more of this too. [0]

[0] https://sketch.dev/blog/our-first-outage-from-llm-written-co...

Why is it such a dramatic statement for Boris to claim that he no longer writes code?

Are people on HN still typing out functions by hand one character at a time?

It would be like a developer in 2020 claiming that he only writes assembly because compilers can’t be trusted. No one is taking that person seriously. If you chose a career in tech you made a decision to work in one of the fastest moving fields in human history. Now it’s time to get over it, learn the new tools and adapt.

> Now it’s time to get over it, learn the new tools and adapt.

No, thank you. I have used the new tools, determined that they aren't helpful to me, and set them aside as I would with any other bad tool. I don't feel the need to let hype take the steering wheel.

Let me guess, you used them 12 months ago?

> Now it’s time to get over it, learn the new tools and adapt.

Exactly. You are free to use openclaw or a coding agent to build a competing bank, hedge-fund, hospital or even a new airliner because the previous ones were built by humans. Surely an AI can do it better by itself.

So why haven't you done it yet?

> Are people on HN still typing out functions by hand one character at a time?

Yes, me. Yes, I tried LLMs for what I am doing and will try again in few months. No, there was no noticeable or clear improvement over doing it manually.

Yes, I am using some LLMs for some purposes but Claude Code had slight improvement, if any, not worth introducing proprietary dependency.

> Why is it such a dramatic statement for Boris to claim that he no longer writes code?

Because we can actually see the disjointed slop that Anthropic produces. And when issues happen, they can't fix them for weeks on end because no one understands what code does anymore, and all of their "hard problems causing issues" they blog about are literally "if we had actual engineers this wouldn't even be an issue to begin with". Like this bullshit they had in spring: https://www.anthropic.com/engineering/april-23-postmortem

> It would be like a developer in 2020 claiming that he only writes assembly because compilers can’t be trusted.

LLMs are not compilers. For a few very obvious reasons I'll leave as an exercise to figure out

> Now it’s time to get over it, learn the new tools and adapt.

If the AI is producing what you tell it to, why are you needed?

You answered your own question... you are needed to tell it what to do. Let's not pretend that someone with prior software skills will be able to produce larger scale and/or higher quality work compared to someone with no experience.

> Let's not pretend that someone with prior software skills will be able to produce larger scale and/or higher quality work compared to someone with no experience.

This seems like a really confident prediction. It isn't true right now, why do you think it will be true in the future? Right now having knowledge and experience is a huge benefit to steering the LLM (it makes dumb decisions all the time still).

>Are people on HN still typing out functions by hand one character at a time?

Well I use tab completion, of course. And I copy-paste snippets from LLM more often than from SO now. But otherwise not much has changed in my career in the last 5 years. Is this different for you?

I'm not fundamentally opposed to code generation, and I use LLMs for some taks, but I don't see myself vibecoding whole pages of production code. I vibecoded a throwaway note-taking app for myself though.

Yes. If you're copy posting code from LLMs you're in the minority of people who are "in the past". Most people are generating most of their code in ~1-200 LOC chunks, reviewing it, adjusting as needed (usually with another prompt) and then opening a PR, which gets reviewed both my LLM and other teammates.

Citations very much needed on the "most" here. I'm in a huge corpo with ~1500ish devs and the large majority of code is still very much handwritten, and we have access to all the AI tools imaginable (it's not forced on us because they treat us like professionals that will use the most appropriate tools to do the job, thankfully).

Our whole team has transitioned to agentic engineering around the start of this year. We've been heavily investing in harness engineering to ensure final outputs are production quality and not slop.

These days it very much feels like the task has shifted from "building the system" to "building the system that builds the system".

It only looks to be trending one way.

C'mon, the LLM/compiler false analogy? In 2026?

It is because HN is contrarian and behind the times.

I work at a big tech company and I don't know a single person that still hand writes code. Most people haven't hand written code for at least half a year now.

I do wonder what sort of bug is making its rounds on HN that people here find this so shocking and unbelievable.

The bug is called "applying actual engineering principles and critical thinking".

The absolute vast majority of people who point out AI's downsides have used it and use it. People who uncritically write things like "I work at a big tech company and I don't know a single person that still hand writes code." scare the shit out of us for a good reason.

Using it is not enough. You have to explore its boundaries, see what it can do when cost is not a constraint. This is only really possible at the big tech companies and well funded startups.

Any eng that is only using Claude Code or Codex or whatever, is frankly not entitled to talk about AI's limits since they are using the most basic harnesses. They literally don't know better.

When I see Claude Code or Codex users on HN talking about how coding with AI is risky, it's like watching someone that has only ever seen a catapult argue about how space travel must be impossible.

> Any eng that is only using Claude Code or Codex or whatever, is frankly not entitled to talk about AI's limits since they are using the most basic harnesses. They literally don't know better.

I really doubt big tech has much better harnesses than are publicly available. Definitely not "catapult vs space travel". They have the same base models we all have access to.

solenoid0937 is not working at any big tech company, instead they are being paid by Anthropic to spread unfounded LLM-hype. They have a history of doing that. See https://news.ycombinator.com/item?id=48270186 for more info.

Ah, my stalker has returned! I was wondering where you were. Yup, I can confirm that Dario personally hands me a check every evening.

It was my impression that a whole lot of products are only pretending to be compliant, and that it's much more profitable to operate like that.

I've worked in fintech for 30 years. I've never seen a product that was intentionally "only pretending to be compliant" with laws.

I've seen accidental non-compliance. I've seen what I would call negligent compliance, where a company attempted to be compliant but didn't meet full, correct compliance (one example I've seen is that a company assigned resources to compliance and forgot to increase resources as workload increased, causing them to be increasingly behind on compliance work), but I've never seen a company that just decided to pretend to be compliant knowing that they were not.

Never seen one or never worked at one? Because gestures broadly

In my experience this is not representative of most fintechs. Of course there are both cases of real intentional noncompliance, and accidental, but by and large it seems like everyone’s trying to innovate within the law.

This makes sense because these companies want to become large companies and contract with large companies. Large companies, by and large, try to follow the law (while trying to bend it to the limit) because they're aware they have a big target on their back and no CEO wants to be on the front page of the papers for tanking a company in such a stupid fashion.

Even if that's the case, I feel like accurately knowing which regulations you're in compliance with and not is would be kind of important from a risk management perspective. From a "maximize profits" perspective (which I'm not saying is good but what you're saying you thought they operated with), you'd want to know the potential gain from ignoring a given regulation and the likelihood of getting caught (along with the cost of the punishment if that's happens). This is the kind of math that I'd expect a finance company to be pretty familiar with, and giving that up for a fuzzy "idk if we're in compliance or not" check seems like a pretty huge liability (unless there's confidence in not being liable for blindly trusting the LLM, which I hope is not the future we're headed for but I guess I can never be totally confident in us not somehow ending up with rules that defy common sense).

Companies that are growing tend towards faking compliance. Many financial rules like pci only kick in at certain scales. So a company growing very quickly will often be behind the curve but will do everything to seem like they are compliant. Then they would hire people like me to come in and make them actually compliant. More often than not, making an effort at improvement was enough to keep the ball rolling.

I think it's the same throughout startup software to be honest. It's just easier to point out when there's clear rules.

Security, GDPR, backups, build pipelines, disaster recovery, most of it will be faked, half-heartedly done once or ignored entirely.

Then there's the more abstract things like scalability, idempotency when integrating with external APIs, error recovery, accessibility, UX, etc.

Almost always that sort of stuff will have been entirely ignored, or there will be a fig leaf over a real mess of misunderstood standards or manual intervention steps.

Startup developers usually have to be generalists as they often wear many hats, so things that need deeper domain knowledge get done to a bare minimum.

Nah. There are for-real consequences to that sort of thing and it's so not worth it.

Early stage startups, maybe, but most of those are three failsons in a trenchcoat. That sort generate more PR than revenue.

In the same sense that selling illegal drugs is a very profitable business to be in, until it isn’t.

Where did you get this impression from?

A worldview built on reading comments from news aggregators.

I've worked on projects in the airline and health industry which are highly regulated too. The regulations can be incredibly difficult to process and implement, and make sure you adhere to everything correctly. I've been involved in multiple scenarios where people have made false assertions about compliance or lack of. I'd still place a bet that the SOA models make _far_ less mistakes than humans.

They might make fewer mistakes, but they aren't evenly distributed. They don't use logic when making mistakes, it is gaps in the training data and now large of a span they have to bridge in the latent space. Just as they aren't smart like humans, they aren't stupid like humans. Don't mistake rate for quality.

Yeah, this starts to overlap with some autonomous vehicle stuff, where I like to say that the rate of errors is not the shape or distribution of errors.

We have long historical experience and innate tools for detecting and mitigating errors made by humans. If we can't apply those to automation, then even fewer total mistakes may end up being a worse outcome.

>I'd still place a bet that the SOA models make _far_ less mistakes than humans.

Genuine question: your top coder seems to be producing the most error-free code from your perspective, has the deepest knowledge of the architecture and codebase, and is faster on the trigger than the others.

But your top coder has proven and verifiable dementia, where they will confidently assume the existence of apis and code that do not exist, mix up the purpose of others and forget other things, and you can't predict when and how they will introduce errors into the system or the severity of such errors.

Are you really comfortable letting this person with dementia generate most of your codebase in the airline and health industry?

I also hope you have an iron-clad agreement that prevents the model provider from doing silent updates because all your evidence of correctness you collected thus far goes out the window in that case.

Another genuine question:

You have witnessed a human coder and the AI you're using make the same important mistake. Assuming you do not have the time and resources to retrain, fine tume, and test your frontier model:

Who would you trust not to make the same mistake multiple times in the future after you have warned them that their job depends on it, the AI or the human?

Your top coder has guard rails in place to prevent him autonomously going free - right? This is how you should approach agentic development with LLMs. Like it or not, we are the final bastion, the gatekeepers. The hallucination thing I think is mostly overblown and from speaking to colleagues it seems to vary wildly depending on which model and harness you are using - always go for SOA. In the last 3 months I can count on one hand where it's done something wrong and that's primarily as I'm operating it with guard rails and giving it context.

>Your top coder has guard rails in place to prevent him autonomously going free - right?

The parent is implying they would prefer an AI when working in the airline and health industry because it makes less errors. Read the comment again.

They have not said, "Hey, I work in the airline and health industry and I'd love to use AI for a couple of the bullshit IT UIs we have as long as we can put guardrails on the AI to stay in its lane."

I asked a yes or no question. The guardrails you can put to mitigate errors are the same guardrails pre-AI for the humans (tests, regressions, reviews). If you were wary of employing a top lead engineer with verifiable dementia prior to AI for a mission critical system, logic implies you should think twice giving that much responsibility to an AI as well.

> The hallucination thing I think is mostly overblown

Can you predict when and how the SOTA model will hallucinate? Yes or no. Can you predict the severity impact of that error beforehand? Yes or no.

>from speaking to colleagues it seems to vary wildly depending on which model and harness you are using

You have partially answered my question it would seem.

> Can you predict when and how the SOTA model will hallucinate? Yes or no. Can you predict the severity impact of that error beforehand? Yes or no.

No, but the same can be said for your colleagues. You might call what the LLM does hallucinations, I'd call them mistakes. I think we have totally forgotten that humans make them all the time and are confidently wrong too.

Your original question, doesn't really get to the bottom of the point I'm trying to make, and I don't really feel it fairly represents the issue we are talking about here. They are not the same things.

But your top coder has proven and verifiable dementia

Dementia gets worse. AI gets better. Nothing matters except d/dt.

For some reason, tons of people seem to be in camps at both extremes. It's either "AI sucks don't trust it!" or "AI is so much better than humans!"

But the most reasonable take, which I'm happy to see reflected in so many comments in this thread, is… use both.

Do an AI pass, and have humans verify, and vice versa. Let the humans drive the AI. Then the unique shortcomings of each party can be covered by the other's strengths.

AI review is never going to beat a fully resourced human review.

It might beat an underresourced human review, on time, efficiency, cost metrics. But on the metric of accuracy, throwing unlimited humans at a problem will still beat throwing unlimited AI at it

That's an irrelevant comparison because cost is always a constraint, so there are not going to be unlimited AI or humans. The question is how to optimally combine them for a given cost.

> Do an AI pass, and have humans verify, and vice versa. Let the humans drive the AI.

You can do that, sure. But doing so negates any improvements in speed the LLM brought. And at that point, you may as well just do it yourself to begin with.

When Google showed up on the scene I found I no longer needed to memorize basic syntax and other such things. If I couldn't remember on the fly, i'd just do a quick google search and move on. This freed space in my mind to instead focus on bigger & better things.

I use GenAI tools when coding a lot, but I do not vibe code. I go through everything it generated, and we iterate. And yes, it doesn't save me a lot of time. But what it does do is free up mental capacity in a similar manner. But instead of syntax, it's more complicated patterns. Maybe I don't remember how to stitch something together, but i know it can be done. Instead of spending the time to look it up and then code it, I just tell it to do it for me.

> Maybe I don't remember how to stitch something together, but i know it can be done.

That's how I use the current AI, too. I never ask them to do something without specifying how it should be done. I ask questions first, use /plan to let the model ask me questions, then I let it execute the plan while reviewing the results. More and more often, I get something close enough to what I would have written. In the opposite case, I at least know exactly how to rewrite the result, if needed.

I observe the same effect as you: while it does sometimes speed up the implementation a bit, it's not very noticeable; however, it frees me from having to recall all the obscure little details up front. Instead, I can describe them, have the model implement them, and then recognize them (and refresh my memory) when reviewing. The effect is that it's easier to start a task because I don't need to prepare as much to execute it. It's especially notable on things that I haven't touched for some time. I know, more or less, how my Elixir projects are set up, but after ~2 years of not working on them, getting back into them had been a hassle - with AI, it's no longer that. I think the biggest difference comes from the AI lowering the cost of context switching for me - I used to have huge problems with that, and AI certainly helped a lot.

Yeah, humans reviewing the AI review can only detect the false positives, where the LLM claims something is non-compliant and flags it for review/correction by a human or another agent. Human review can’t find the false negatives (true deficiencies not flagged) unless you do a full audit yourself to find whatever deficiencies the AI missed.

>But doing so negates any improvements in speed the LLM brought.

We could do with less speed.

I feel like you're missing the point that it's more thorough to use both. Speed isn't the only factor that matters.

This makes sense, but a logical next step is to have one AI write code, and then have another AI, instead of humans, verify it.

Or are current AIs too similar for that to be fruitful?

This is commonly known as "LLM-as-a-judge" and anecdotally multiple people I know who write code using OpenRouter or using multiple models say it's surprisingly effective. It's strange that there don't appear to be any major papers on it since ~early 2025, which at this point is basically ancient history.

not according. to my experience.

regulation questions. even the simple ones, AI gets all the time wrong. it wasn't Mythos, but other models like opus.

I can adjust the view on this topic if/when we get access to mythos.

> I'd still place a bet that the SOA models make _far_ less mistakes than humans.

Well too bad, the problem is that they also produce things much faster than humans so errors will compound quicker.

IMHO even if we are using auditing tools I believe we must use deterministic tools for critical analysis like this. Such rule and pattern based systems may not scale beyond certain point but they can be accurate.

> anyone who's using them to actually ship product without a human actually digging into it is opening themselves up to a world of risk.

Maybe it's just me, but it seems that companies will happily take existential risks to get a better bottom line short term. Either you're too big to fail or you've already privatised any profits and subsequent losses (due to the risks becoming manifest) are socialised. The motor industry seems to be particularly egregious in this aspect, but also the food industry, construction industry etc.

Seems to me even governments make the same choices in many ways - cut back health-care, policing, education, public transport and let the next government deal with the consequences.

> Seems to me even governments make the same choices in many ways - cut back health-care, policing, education, public transport and let the next government deal with the consequences.

Definitely - defund the police was an astonishing rallying cry that made the communities it pretended to help much more dangerous for their residents.

But the opposite is far more likely, and far more destructive long-term: it's easier to buy votes by spending more on social programmes and rack up debt and/or inflation to cover it, and then spend even more to fix that problem for enough voters that the people paying for it all can't vote it away, and the people who vote for it over the decades just don't understand why their pot feels so uncomfortably warm all of a sudden.

I use Opus 4.8 and GPT 5.5 and haven't suffered from hallucinations in months. But we also put a lot of effort into our harness.

Opus 4.8 and gpt constantly hallucinate stuff as well. If you haven’t encountered or caught it that’s something different. Of course these days it’s mostly confidently asserting a wrong thing.

They said that they hadn't "suffered" from hallucinations in months due to effort they put into harness engineering. It's not quite the same thing as saying hallucinations are never happening for them.

I find hallucinations happen for us with those models, but we've worked on baking in guardrails and fact checking against sources of truth, so that it's less of a problem.

We're engineers. We just try to engineer out the failure modes.

I primarily use Opus for a Lisp-like DSL codebase (non public, closed source) and it genuinely has _never_ hallucinated. All it pulls from is BNF, language spec + examples. So I have no idea how people are getting it to hallucinate on _popular_ languages.

I think you forget that they really are stochastic (I'd wager it has hallucinated things to you that wasn't important so you missed it, or you've just been very lucky), and the people you're arguing with are forgetting that there is significant difference in when and how often even frontier LLMs hallucinate. I catch claude every now and then, but isn't the measured hallucination rates down in low single digit percent for claude now?

Sometimes the harness can only be a human.

And this is fine. Developing new software with a really smart intern is the same, you, as an expert, need to bring your experience/expertise on the table to have everything right. Because experience needs time.

Those models simply don't hallucinate if you use them properly in any form. The only way they _might_ hallucinate is if you use the web based chat interface and give them zero context.

Ah the old “you’re holding it wrong” argument.

Sorry, but we aren’t at the “press a button to build a game/app” yet.

you should rewrite this comment with chatgpt if you do not want to be dox

https://www.tomsguide.com/ai/ai-can-now-identify-anonymous-i...

False-positive rate is so high with Mythos according to friends and other reporting I have seen.

The original Mythos release used ASan to filter false-positives so it was able to maintain a good FPR, but when Mythos moves into domains that don't have a readily available oracle to help filter hits, the result is a deluge of false bullshit.

3 years max. Maybe 5 if you are lucky.The models will continue to improve. The exponential gains in compute efficiency that have been ongoing for 70+ years will continue and that will result in even smarter models. There are dramatic hardware changes in the pipeline.

But really that particular issue could have been solved by literally just telling it in a markdown file or instructions something like "verify all facts or compliance requirements with web search and include citations in responses".

This is akin to “don’t make mistakes”

“Verify all facts and compliance requirements” leaves enormous holes even if you assume the LLM has a concept of facts and requirements (it does not).

What facts? What requirements? For what industry? For what subset of that industry? For what country or countries that you will be doing business in? Are these current “facts” and “requirements” or is the LLM referencing a dusty article from 1992 for which the subject matter has been radically overhauled?

In my job I regularly see small but incredibly important mistakes like this lead to major issues. Some of those are human driven but increasingly the defense of the person responsible has turned into “Claude said it was fine though!”

> “Verify all facts and compliance requirements”

No. This is a disasterous instruction. Not only is it vague, but it's also meaningless. When giving instructions to an LLM your prompt must be concise and exact. Tell it _exactly_ which requirements need to be followed, ideally have it write or (preferably) pass audited tests to enforce these requirements. You also need to provide it with a hard source of truth it can rely upon. Instead of saying "verify facts", you're better off by saying "... make sure [whatever you're doing] matches with data at X.Y.Z, verify by running [instruction/command/program]"

I think you might have meant to reply to the parent comment.

Especially in cybersecurity.

If someone can’t distinguish between the two then I honestly wonder what company would be comfortable putting them anywhere near a regulated or security-sensitive workflow especially from someone one that condescendingly views their own jobs as a daycare for people seemingly beneath them.

It can make mistakes and will sometimes, but what he specifically mentioned was a case where it did not pull up a reference that it needed. So using a web search tool effectively would make a big difference.

It still does not rise the standard he requires which your response indicated would be easy for the model to achieve with a simple prompt.

Additionally, using a specific tool does not suddenly give the model common sense enough to say “this piece of information doesn’t answer the question of whether this solution fits in this specific industry at this time in this place”.

A web search tool to pull up the law that is relevant?

Well, you wouldn't just give human a task "verify all facts and compliance requirements" and expect it to end well either, no?

If I was working with someone who had experience in the specific industry then yes, that is in fact what I would do.

If I plucked a random passerby and gave them the task then no, I’d find myself detailing out every specific to them.

You’re equating the LLM to the least qualified candidates. I don’t think your argument is communicating what you intended.

of course not, nobody experienced at their job would/should be saying that and expecting it to be flawlessly followed through especially cybersecurity.

feel like the parent you are replying to literally views their place of work as a daycare which is very condescending

You’ve managed to contradict yourself between your 1st and 2nd sentence. I’m not sure what point you’re trying to make.

The argument is regarding LLMs and domain knowledge.

> 3 years max. Maybe 5 if you are lucky.The models will continue to improve. The exponential gains in compute efficiency that have been ongoing for 70+ years will continue and that will result in even smarter models. There are dramatic hardware changes in the pipeline.

I remember hearing that 10 years ago about self-driving.

60 years ago about flying cars, 40 years ago about cold fusion, the list is long.

We need a lot more basic research into LLMs and also a lot cheaper hardware.

The current batch of LLMs will turn a lot of fields upside down, but not to the tune of $3tn or whatever crazy amounts are being invested right now.

I mean basically you and I are effectively living in parallel universes. Waymo has been running for years, and there are other services including in China and Tesla which is not 100% there but actually very effective.

And the thing he complained about is fixable with a web search, and AI does programming and office work today. So, it's already here. It's just a question of degrees.

Waymo heavily relies on real humans to get their robots unstuck. They also rely on extremely detailed mapping data, which is why they're only in a few cities.

Tesla has been a couple years away from FSD for, what, like ten years now?

If you scrape off the glitter, you'll find a lot more duct tape and wire than you think.

"Just 2 more weeks guys, and AI will be able to do everything!"

Ah yes, the magical equivalent of "you are a senior software engineer who writes bug-free code".

IME people would benefit greatly from the process, albeit tedious and time-consuming, of testing out the same prompt sequence/session with the exact same model multiple times. It becomes clear extremely quickly how capable but unreliable and inconsistent a model can be even when given the same context. If you have ever completed a long, complicated task with an agent and then lost the session and tried doing the same thing again from scratch you may have had the experience of seeing the subtle changes that come up in the model's thinking which lead it to accept or reject certain paths and ignore or incorporate prompt instructions like the one you've provided.

Change the temperature to 0 and it will be more consistent.

Stuff like that is risk tolerance... its not strictly codified and its more akin to probability. Different companies at different stages, in different industries will all interpret their risk differently... how will a smarter model improve that?

The classic 3-5 year window for a new technology that is uncertain and requires just a few more breakthroughs to get there...

written with confidence too. I'm amazed at the levels of confidence people have in predicting the (unclear) future.

LLMs learnt to be confidently wrong from us.

Like full self driving!

In some sense, you should still act on this, since if an external auditor relies on the same stack, it'll still cause you headaches.

The models can change at any time and behave differently.

> the code in question had already been reviewed by human counsel

You sure?

> it had of course hallucinated what the regulation actually required

Did it do the correct job once you put the regulations doc(s) in the context?

What I usually do when in doubt is challenge the AI. “Please quote the section of regulation the product is non compliant with”. It usually admits it hallucinated the whole thing.

It sometimes says that even if it hasn't though, so like everything with LLMs, you can't actually rely on that.

You should just hit retry, usually they either latch onto the correct part of the latent space (surprisingly often) or they admit they don't know (or call a tool, depending on the model).

Have you added "Make no mistakes" to the proompt? Mythos can't go wrong then, must be a skill issue.

its shocking people don't realize you're being ironic

AI cannot fail, it can only be failed

My current favorite in that area ( because I saw it in the wild ) is:

"Make it better" with no additional or reasonable previous explanation of what better might mean.

"AI will figure it out" not for pattern extraction, but for a full blown analysis with equally generic prompt all confidently stated by an executive telling people working it how it works

If you talk to it like a programmer talks to a computer, it works a lot better.

So the question remains if non-programmers will adapt, the LLMs will accept wider range of input styles, or .. its just another abstraction layer for devs to use.

I've observed this in the wild where someone is iterating with an LLM and giving it only negative feedback. For example responding to edits with "don't make it blue" rather than "keep the existing button shape, and change the color back to green".

The LLM doesn't really come back the way a human would and say "so what color do you want?".. it just, guesses. Now abstract that to more complex tasks.

I realize they’re being ironic, it’s just a poor contribution to an otherwise productive conversation.

what am i missing?

you take a spec and create tests, every little thing

you use another ai to verify these tests against the spec

you review the tests vs the spec (at one point human review)

you put the tests off limits to change / wall them.

you let the ai write the software that fulfills the tests.

there will be some gaps where you repeat the cycle above

if the tests fulfill the spec, the code will fulfill the spec

>you take a spec and create tests, every little thing

A spec detailed enough and unambiguous enough to be translated into machine execution deterministically is called code.

Unlike a compiler, AI can build with a spec that is not detailed enough or unambiguous enough: It does so by filling in the gaps with educated guesses.

This is safe if and only if you take the time to later read the output, understand what its guesses were, and judge wether they were acceptable. No AI can do this for you because the truth lies in your original intentions, which it does not have access to.

The jury is out there on how reliable and time consuming this is vs writing the code yourself; it is not immediately obvious that is faster or requires a smaller cognitive load.

Code is not a spec. It's an instruction set. It can be a spec if you try hard but that's not an inherent property of code. For example you can write code to be a compiler..that makes it a spec. But hello world is not a spec.

As for whether or not LLMs can write unit tests. The answer is yes.

Hello world is a spec. The spec says to produce the text hello world on standard output.

Try running it without a compatible ABI. See how far you get.

If each step requires micro-steps iterating with an LLM with human review to prevent hallucinations creeping in.. at some point you might just be better off letting the human do the work.

Particularly as tokenmaxxing has ended and people are being charged more economic prices. If the pricing 5-10x the way Uber,etc did on the path to profitability.. even more so.

> If each step requires micro-steps iterating with an LLM with human review to prevent hallucinations creeping in.. at some point you might just be better off letting the human do the work.

I mean for a lot of spec code people define the API signatures and let the llm run with it which is an excellent tradeoff.

IME, regulatory compliance is something you are rarely able to test for in a nice little box or with well-known suite. So there's no easy "this complies" in many situations, no matter how many lawyers, compliance officers, and llm's you run it past.

so, whats the difference to human engineering?

other than there are "internal micro feedback loops" during development?

I walked down that path for a few months. The more you constrain LLM's, the more underhanded they behave in order to produce something that satisfies all the constraints.

Doing the above doesn't actually make the model smarter, so, if it couldn't get to correct code with fewer steps, then the light you see at the end of the tunnel is an oncoming train.

This is such an abstract principle that the principle itself cannot be refuted. The plan sounds fine on paper. "Just iterate bro". But it entirely depends on what rational agents you put into the system. Obviously, if I sub in a 5 year old child everywhere, this loop breaks. Humans and AI, sometimes one is better than the other at certain things, we're still learning.

The only way to test this is to test it out, in real life. Sometimes people see results, sometimes people don't. Note that yes, I am including the entire iteration process - even after iterating, people still don't see results with AI.

I have had both positive and negative experiences with AI, over multi-week projects. But apparently on hackernews, anything positive about AI is proof that AI is superhuman and taking over, and all follies about AI are lies by stupid humans who secretly have psychological dispositions to fear AI. Sometimes the AI genuinely isn't good enough. Are we not allowed to say that now? We might not know why, but it's just the truth.

The other solution is to formally analyze the entire space of possible actions the agent can take a priori. Then yes, you can definitively say whether or not the principle breaks or not. Can you, though? Can you give a formal specification for the space of possible actions for AI and show that your loop never breaks, or breaks less than humans, or any other sensible criteria? If not, then you can't just give an abstract principle and start making inferences from that.

It’s impossible to write a spec that’s not ambiguous , complete and correct in natural languages. Thus prompts will always generate unreliable software.

Is that all that Mythos did?

Did it find any real potential issue, optimization/simplification opportunities, or sparked any thought-provoking discussion within your organization?

Or was it purely a net negative experience?

Read their comment. It's a negative anecdote surrounded by them using genAI all the time.

You're the only one coming away thinking there was a net negative experience.

In regulated industries none of those matter if the tool invents compliance issues or breaks compliance.

The only thought-ptovoking discussion should be "why the hell do we have this stochastic parrot anywhere near out codebase"

I think that what technical people fail to understand is that a lot of the time, "compliance" is not the same as a binary compiles/does not compile. For a lot of rules/regulations, compliance means "making enough effort that legal is willing to back you up".

A system which will just randomly decide to give the legal team reasons to not back you up is:

* A system whose output will get brought up in lawsuits and make legal's job harder.

* A system that will make the dev team perpetually chase its tail while it oscillates between the several different valid interpretations of the rules.

Odd take. So if it identified 17 real gaps and helped fix them, the fact it was wrong about one gap, and the appropriate humans caught it and no harm was done, the whole thing is useless?

Not saying that is the situation, I don’t know. But if “one error is too many” is your point of view… do you think the humans in these orgs are 100% perfect 100% of the time?

> So if it identified 17 real gaps and helped fix them, the fact it was wrong about one gap, and the appropriate humans caught it and no harm was done

How many gaps have humans not caught?

> But if “one error is too many” is your point of view

Yes, in regulated industries "one error is too many" is the only right approach.

Yes, humans also make errors, and there you have a range of options: from tracing and finding the causes of the error (and tightening processes) to literally jailing those responsible. Your hallucination machine will happily "identify" 17 gaps, and create 34 more. And no, there are no processes to make it better. The "make no mistakes" incantation will happily be ignored for obvious reasons, regardless of how many forms of it you throw at it.

It doesn't seem like you're engaging with the material circumstances described above. What does it mean for a human to not catch that a part of a codebase is actually compliant with regulations? What does it mean for the hallucination machine to create 34 more gaps when it doesn't appear to have more than read access? How would it not be useful to have a machine that identifies 17 real crimes that your highly regulated business is unintentonally committing even with a 90% false positive rate?

Isn't that a net positive though? (not sure about the cost human and tech cost). I'm guessing that without using Mythos, those conversations would never have been had, and confidence in the compliance of the product would've been lower.

I love using AI tools as casinos. It's epic in helping to forge ideas and kickstart thought processes. You basically have the entirety of world knowledge at your fingertips to have a pint with.

> I'm guessing that without using Mythos, those conversations would never have been had, and confidence in the compliance of the product would've been lower.

The conversations had already been had and the product made compliant. Mythos just pulled new rules out of its ass and of course the product wasn't compliant with those. So they do a fire drill and find that to be the case at great expense.

Yeah you can frame it as "more checking is always better" if you wanted but that's just the same old "other people's resources are valueless" slight of hand we see on everything. It probably was mostly wasteful work.

There's a chapter in Simple Sabotage about how to undermine a white collar organization from the inside. One of the key tactics is to hold meetings that revisit decided upon points, and to invent unnecessary process / checking.

So, in this case, the LLM's behavior was equivalent to the behavior of the resistance during WWII.

I think that book should be required reading for all engineering students.

your parent:

> the code in question had already been reviewed by human counsel

They cant read all comments they comment on...

> It's the expertise of engineers on the team that push it back on track.

But how are you so sure your colleagues are not more "expert" than you? Prior LLMs there was room for very good engineers and mediocre engineers to work together in 99% of the companies out there. With LLMs, only the "best" engineers will survive, because nobody needs mediocre engineers anymore.

This being HN, I imagine every engineer reading this thinks they are in top the 10-5% of their company/city/country, and therefore they think they are not "mediocre" engineers that can get affected by the introduction of LLMs. Statistically, they are probably wrong. So, it's all about ego. Chances are you are not a rockstar and LLMs will eventually take over your job.

As usual, the only winners here are corporations and executives. Most of us are the last monkeys in the chain, and so we'll get screwed.

The corporations and executives are already winning if you swallowed the concept of 'rockstar' engineer. Sure there are more and less experienced engineers, but even interns can and often do provide good input and spot mistakes made by seniors. The 'rockstar' engineer at most tech companies simply equates to the somewhat autistic guy with a brown nose who's working 15 hour days for a pat on the head from management (and making many mistakes in the process).

>The 'rockstar' engineer at most tech companies simply equates to the somewhat autistic guy with a brown nose who's working 15 hour days for a pat on the head from management (and making many mistakes in the process).

I love it! And posts on HN about Big Ideas and uses corpspeak to justify driving people to long hours. The engineer who's picked up talking points of his employer because he's well-paid and trapped on the spectrum, making it hard to comprehend a life of Play outside of work.

For the most part there aren’t 10x engineers

But there are certainly 0.1x engineers

The study that defined the 10x engineer defined him as 10x as good as the worst engineer. If there is a 0.1x engineer, and a 1x engineer, that 1x engineer is the very definition of a 10x engineer.

I've long thought a 10x engineer is one with just the right amount of analysis paralysis - not too much or too little. It's not that they're 10x engineers, it's that everyone else is 0.1x due to a confluence of reasons. And the ones we call 0.1x are 0.01x.

Some famous examples of a "10x developer" state: Linus Torvalds writing Git, Brendan Eich writing JavaScript. Somewhat less famously, I get that feeling often when I stop thinking and start doing on an electronics project, a wooden shed or even a cosplay. Every hackathon ever, same principle - stop thinking, start doing.

But it's only a 10x state if you're doing the right thing, otherwise it's a -10x state, and that means you need to have already done the right amount of thinking and have a good intuition for what you're trying to do. (As long as you can recognize a failed experiment and revert, risk of being -10x isn't that terrible)

There certainly are 10x engineers just that they get most of the x from turning down bad ideas and saving work.

and -10x

This is such an obvious observation that I’m surprised to find it often missing. 0.1X is nothing compared to the destruction (ie negative X) you can do with the right combination of recklessness and managerial pressure. Definitely happens with engineers. Perhaps even more with PMs.

Even if we forget "rockstar", there are certainly different levels of engineers. More experience doesn't automatically mean better either. That is not to say experience doesn't matter. It matters quite a bit. Sure , good interns can sometimes have good feedback or spot mistakes. But not consistently enough.

All of this to say that it's not just experience that makes one a better engineer.

Experience is one of the only objective signals we have, but you're right it's not the only one. I've seen plenty great junior engineers and interns, and plenty of incompetent staff/principal engineers.

True, it is the only objective signal, but as you have observed as well, it is also a very poor one.

> because nobody needs mediocre engineers anymore.

This is giving too much credit to LLM. I think LLMs are great and it is incredibly useful both in personal and professional settings. However, it exist on a separate plane than human workers in the tools category.

Sooner or later, people will find out that LLMs only overlaps with existing human hierarchy (e.g. junior dev X%, senior dev Y%, etc), but almost never 100%. If it was 100% to a certain position, you are probably using the humans wrong to begin with there - since humans have one of the most priced thing that I don't see an single ounce out of LLMs: initiative

It's hard to show initiative without a pulse. Most agents don't have that (yet). But can't be too hard to build.

Well I think that's the next trillion dollar question, if you can figure it out.

We don't need to know the entire knowledge base of mankind to want or know what to do next. It points to an entirely separate architecture than LLM.

> With LLMs, only the "best" engineers will survive, because nobody needs mediocre engineers anymore.

I don't think this is true.

A good engineer doesn't have infinite throughput. In my opinion the best engineers should be constantly bottlenecked because they solve difficult problems. They don't have time for grunt work. Every company needs less than perfect engineers, AI assisted or not.

Perhaps that's true of the top 5% engineers. But the question is if the top 40% of engineers can replace the bottom 60%.

The LLMs don't need to be perfect, they just need to be good enough so that the cost of fixing their code is lower than the communication overhead and the 'lost in translation' overhead from delegating tasks to mediocre engineers.

I am sure the tech industry will try to further reduce headcount. The question is what jobs emerge. We already see companies laying off, just to hire "AI interns".

If AI + lower skill engineers turn out to be valuable, it also becomes more attractive for smaller companies and startups to do software or expand / inhouse software. Which is actually a good opportunity to level up, because many devs don't become great because they miss the chance to take responsibility. Which is harder in smaller companies.

> With LLMs, only the "best" engineers will survive, because nobody needs mediocre engineers anymore.

LLMs are going to show that there's a huge divide in "engineers" between people who love "coding" and people who like "engineering".

The group of people kicking and screaming the most are the people who love code and don't want to see their coding go away.

These are typically the build vs buy folks. "We can't use anything anyone else wrote, I can do it better..."

What do you think Staff level engineers do? They don't sit around coding all day.

Writing the code is just something you had to do in the past to get the job done.

What you get paid to do is "engineer". The two are related, but they are separate. Coding is a very small part of the average engineer's job (and almost none at staff level and above).

And yet the vast majority of engineers think that the world is going to end if they aren't spending most of their time "coding".

I disagree with most of this. I'm kicking and screaming pretty hard, and yet I'm not one of the "I can do it better" folks. My whole career has been in open source. I'm all about choosing the right tool. I'm also tech lead on a team of seven, so I'm not writing a lot of code anyway. What I am doing all day is sending AI-generated code back to be rewritten, rethought, re-architected. I'm starting to think we would get more done if nobody on my team used Copilot.

Why is everyone acting like this is unknowable? Can’t you just look and see what the team’s output was like 2-3 years ago? Of course there might be long term risks that have a time lag and are thus not visible yet, but it shouldn’t be a complete mystery

Would you explain this - " I'm starting to think we would get more done if nobody on my team used Copilot." ?

Interesting to see if you why you see it destructive. I can certainly see that a lot of the times, copiloted solutions are working but less than ideal

> What I am doing all day is sending AI-generated code back to be rewritten, rethought, re-architected. I'm starting to think we would get more done if nobody on my team used Copilot.

Sounds like a "your team" problem, and not a tool problem...

I hate to break it to you...

If your team can write crap in Ruby, they can write crap in Rust, too.

And if they can write crap with AI, they can write crap without it, too.

Very well said and if you look at some of the other threads on hacker news about why people don’t like AI it specifically because they like typing and coding

The majority of my time is an engineering manager has been teaching “engineers” how to actually do engineering with any kind of rigor

The number of engineers who have an absolutely no theoretical structural or system basis for what they’re doing is the vast vast majority

Exactly. Same with tractors. Once they arrived, nobody benefited except Big Tractor.

Famously a net loss for humanity.

At the very least _big_ tractor (John Deere) is a net drain on society

Agreed, this is the fallacy of current discourse, the guys getting fired due to AI improvements are on the other team, "nothing will happen to me" seems to be the current mindset.

I have seen teams being reduced in size due to offshoring, move into cloud, move into SaaS and iPaaS products, this is the next step and no one is safe.

don't agree with your logic at all - agree with your conclusion. your logic is deeply flawed - a) you've to justify mediocre engineers not been necessary, the mechanism they are pushed out by LLM matters. b) relative ranking is useless - it's the difference between absolute ability and that is close for say top 50% most likely. This might mean half of us will lose jobs, and half see compensation pressured but it's not as certain as your argument make it out to be.

Well almost 70% of the developers in the industry can't write a fizz buzz.

But, besides coding skills (which some possess), the engineering, social, and business ones are close to non existent.

0% chance this is true as I work at a F500 where I can assure you that probably every developer can write fizz buzz.

I would bet my life and everyone I care abouts' lives that much more than 70% CAN write fizz buzz.

Did you pull this 70% out of your ass or from some other place? It's quite obviously not reality.

https://blog.codinghorror.com/why-cant-programmers-program/

There was also another study I cannot find where 56% of engineering graduates struggled to write a fizz buzz.

I think people highly underestimate how long is the average developer, closed in their bubbles of mostly well established software teams that forget that for each of them there's 10 software consultants in southern Europe glueing APIs with trial and error on Java 8 monstrosities.

The link that you gave does not show this 70% number anywhere? Also, the link is from 20 years ago. I don't believe the real number is anywhere near 70%. Maybe 10% or something.

Mate, you're on HN, you live probably in a bubble made of better software developers than the overwhelming majority out there.

Unfortunately every software related industry is embracing LLM/Codegen. Your banks, fintechs, insurance. Everyone. Your concerns are the same I'm having, yet it's regularly dismissed or hand-waved away as "don't worry about it the delivery velocity/ROI is worth it"

It's not so much about velocity or quality, both of which LLM do (or will) provide.

The real question is about accountability and liability.

When a major data leak is going to happen, who will they sue or fire ? That is the value engineers provide. They understand, confirm, and take ownership.

This is what I'm wondering too. We've signed a confidentiality agreement with all the big players (as I'm sure all other companies have done), which is supposed to ensure our data is both segregated and not used for training. I don't trust these companies not to do just that; their business is in taking what we have and training their models.

Yeah, I always wonder if they do some type of obfuscation and transformation on the private data and find a way to backdoor the info without technically using it directly.

I wonder if there's a way to include data that's so unique you can prove it was trained on and sue later

Unique data like that is unlikely to have any impact on the learned/final weights after training. SGD, Adam and the other hillclimbing solvers abhor jagged edges from "novel" trade secrets and the like. Unless it turns out everyone had the same secret genius idea (and it became a pattern to learn).

Ostensibly, due-diligence should not change. But people are lazy, just as they've always been around testing/QA/definition-of-done.

I'm not even certain that laziness gets them further along than it used to; I think it's that people have not had their overconfidence painfully corrected yet. Behaviors will re-align pretty fast when people realize that no, they're not going to get away with just pressing a button and saying everything is "good". That is happening right now.

Just having this discussion with someone about AI in healthcare and how issues are going to be handled.

If a nurse does something incorrectly, they can lose their license. Ensuring that nurse will never be a nurse again. There is a very clear path of accountability and very clear ways to mitigate it.

For instance, if a nurse is drunk and you recognize there is a pattern of people showing up drunk, you institute drug tests and breathalyzers and move on.

While we probably won't have LLM's autonomously performing procedures, they are 100% parsing documentation, reading lab results, making suggestions, etc. And right now, the burden has been placed squarely on the clinicians themselves. It'll feed them them the data, ask if they approve/agree, and then essentially wash their hands of accountability. Let's say an LLM starts incorrectly reading lab results, how is that fixed/remedied? A prompt update? Additional safeguards? Adjusting the temperature? Changing a model?

This is a far different type of engineering that still feels pretty new. Granted, I'm still an amateur in this space (I use Claude Code a decent bit), but it feels really opaque to me right.

This question has been easily answered by many companies.

You, the IC, the developer prompting the code extruder, are ultimately responsible for its outputted code and its behaviour.

You may feel pressured to push out thousands of lines of code a day. You may see those thousands of lines refactored several times over the lifespan of a merge request. You may be asked to do this continue this in the long term with all the mental fatigue that entails.

When it's too much for you to sustainably deal with and you turn to using LLMs to review the code, that will still, presumably, fall on you at the end of the day.

The output is your responsibility.

> When a major data leak is going to happen, who will they sue or fire ? That is the value engineers provide. They understand, confirm, and take ownership.

This goes for serious incidents, disasters, outages and security breaches.

If there was an investigation and the answer was "a piece of software was vibe coded with AI" why would anyone trust the software vendor after that?

When has any company ever faced consequences from atrociously bad code leaking data or negatively impacting their customers?

Even Solarwinds is still alive.

EU companies are judged guilty of negligence because backups were not totally disconnected (even though distant site) and ransomware did destroy them.

So that is starting to dig deeper than a plain mistake. I guess we will soon-ish witness the first AI slop trial going on, this will be interesting to follow

Knight Capital

Don't worry, we can throw in all in 55 gallon drums and dump it over a cliff when the time comes.

Are banks that concerned about velocity? Because moving fast and breaking things in the banking sector can get extremely expensive. It's also not a who-gives-a-shit industry like operating a taxi service or hosting images, but a very tightly regulated sector.

I might have been a bit broad with the brush. I can't speak for banks, but I can speak for the the fintech/money-movement space (e.g. Remitly, Wise, Revolut).

It's a race to get first-to-market for backend integrations/features. It's given rise to a culture of "move fast break things" where safety is only for some core features, but absolutely not for the constellation of other services we provide. Failure rates have increased almost a percentage point since Codegen/LLM adoption was mandated from up top.

You would think regulators would be on top of this, but our industry runs on all actors "self reporting" their outages. Most don't unless they can't hide it (>1h)

'Keeping up with regulations' may as well be a separate field from the core stuff. It has the same pressures as any other development effort. Managers will want the integration to the KYC service LLM'd as quickly as possible.

> Are banks that concerned about velocity?

Yes

Not in the universe where I live. Having worked in a variety of web tech, and then working at a fintech with a partner bank, traditional banks move incredibly slow compared to nearly every other tech company out there, and for good reason.

I work in a large, established fintech and we're in no rush to shove LLMs everywhere. We have access to all the AI tooling we'd ever need with basically no limits, but AI or not, you're responsible for what you ship, so most people are less YOLO about the whole thing. From the quarterly surveys, most people in the co view it as a slight productivity boost but nothing dramatic, and most features we're releasing are still human reviewed anyways.

Norwegians have a saying: “Den som er ferdig utlært, er ikke utlært – men ferdig.” Meaning if you are finished with learning the one that is finished is you. Typical scandinavic hard cold truth…

I understand the frustration of spending years nurturing a skill and then seeing its value decline.But this isn’t really an LLM problem. The same thing happened to factory workers, typists, draftsmen, and many others before. The technology changes, but the underlying issue is the economic system we live in, where the market can suddenly decide that something you’ve spent years mastering is worth much less than before.

LLMs are not creating that dynamic. They’re just accelerating it.

Reg PRs - for the ones with complex requirements what I am seeing is that time to initial PR is very short, and a ping-pong between the reviewer and developer begins, because in my cases (not all) the developer vibe-coded parts, and they didn't really understand the requirements deeply or their code, and it takes multiple iterations for them to fix it. You can argue this is a human problem but this is the net effect I'm seeing.

I am not sure but for complex cases it seems to me that the earlier sum of moderately long PR time + moderately long review time has been replaced by very short PR time + even longer review time. I am not sure if there's a net gain in these cases. Sometimes even if the code is functionally correct, it's verbose enough (e.g., too many intermediate functions) that I think they will impact future reviews.

> Wut? I pilot LLMs all day but there's no way in hell I'd agree to be at the helm of a finance product.

Dunno how much longer that is going to remain true for your specific employer - all the fintech companies I deal with personally have had some sort of AI account for their devs since last year.

Even places like jane street have employees posting blogs (one of which was on HN frontpage about 60m ago) saying they mostly direct agents.

How long do you think your specific employer is going to hold out?

Sorry if I was unclear. I don't work in finance. I do work with agents. I think expert engineers in finance who are guiding agents are adding a lot of value because of their knowledge of finance. Because I lack that knowledge of finance, even given access to agents, I would not accept a role guiding agents in a finance company because I wouldn't be able to guide the agents well and my/our output would be bad.

Yeah I'm constantly shocked at how simultaneously smart and dumb Opus can be. It can tell me a LOT about my codebase but it will miss very critical clarifications that I begin with. And when I call it out it obviously remembered it, it just ignored it.

> That first pillar is still there. Maybe the author isn't aware of the impact they have, but I know, with the evidence of reverted PRs, that when I step outside my area of deep knowledge I can no longer call BS on the agents. Our most capable agent, with access to the same kind of distributed systems the author talks about, is regularly wrong, frequently myopic, and just outright dumb constantly. It's the expertise of engineers on the team that push it back on track.

I'd posit there's another layer. You have domain knowledge, certainly. But more valuable still is the wisdom to find more.

Anthropic and OpenAI can stick financial regulations in the training data all they want, but the AI systems will never learn to anticipate the future, or reach out to clients, partners, or regulators in complicated situations.

> AI systems will never learn to anticipate the future

Citation needed. I don’t see any reason these systems shouldn’t be able to speculate; indeed some would say that’s all they do, even about the past.

You pilot LLMs all day but that might not last.

A lot of companies are investing money on “ai factories” that are join to automate a lot of software development (that is, steer LLMs) on the basis of jira tickets (or linear/trello cards or whatever).

Funny you mention that, since that's what I pilot LLMs to build :)

I agree with this experience. LLMs are great and save me a lot of time, but they need frequent nudges to avoid going down a completely wrong path. I just don't feel like the management dream of "every engineer has 3 agents working for them full time" is quite a reality yet. I'm not saying it won't get there, or that I feel secure being a software engineer until I'm of retirement age, but I also think it's important to understand the limitation of the tools. You do need to know your codebase. You do need to iterate on small chunks of it at a time. You do need to carefully understand every line of code you're putting into production. LLMs are amazing at generating a lot of proposals, but you need to carefully consider each one.

Most surprising to me about the article was the desire for OP's company to use AI for design docs. I feel like AI-generated design docs are some of the worst -- basically treating English as a programming language. They aren't enjoyable to read, and they often miss the forest for the trees. A human written sketch explaining why we're here and what we're working towards is still meaningful and important. If you want code-level details of every decision and algorithm, we have code for that.

I have mixed feelings on whether these documents are useful LLM inputs. I did a project where I carefully paired with Claude Code on producing a specification that another model would actually implement. I'm not sure it saved me any time, and it was very un-fun. (I kind of blame Opus 4.7 xhigh for this. It ain't speedy.) I feel like I can nitpick code to get exactly what I want, but defining exactly what I want an auto-mode LLM to go and do, in English, is much more difficult. I don't think the PLAN.md I generated would have been useful for a human trying to understand the system (too verbose), and Claude Code still made its usual mistakes that I have reminded it a billion times not to make (t.Context() in tests, not context.Background()!), so I'm just not sure it was worth it. I would say I probably wouldn't do it again in the near future. A rough sketch to get humans on board and to get the high level details worked out, written by hand, and then pairing with the LLM on actually typing in the code seems the most productive to me. But I do try to go outside my comfort zone once in a while to test the edges of these tools. They are very impressive and are worth a lot of the hype. (I know I will never write a YAML file again. I hate it more than anything, and Claude is amazing at it. But I worry I wouldn't feel the same way if I hadn't already had 8 years of k8s experience.)

An LLM agent will only be as good as the environment it operates in.

If you build your environment to be specification based, you have to make sure you have good specifications. If your "memory solution" uses freeform markdown notes, you already lost from the start.

Also choose languages with good unit testing built-in, and languages with unified code styles, and unified toolchains. If you use C++, assume that there's a million ways to build your algorithm. If you use JS, assume 10 different build pipelines. If you use java, assume bloat by dependency hell.

LLMs mimic the ecosystem's variety and variadicity(?). Languages like Go shine so well because it's a very opinionated language, where there's only one proposed way on how to implement things. And that's a good harness to begin with. LLMs are like children on the playground. You have to build better rulesets and fences to make them behave how you expect them to.

Also, check out qwen3.6 coder and heretic models. 30b is the sweet spot for coding and unit testing. For planning and designing, gemma4 is pretty good.

> I pilot LLMs all day

Love the metaphor. Planes are sophisticated machines capable of auto-piloting, but humans are still needed to ultimately pilot the beast.

What about autonomous drones then? They are not planes?

The real reason we have pilots in commerical jets is that they are a failsafe because a mistake in your automated system can kill 100+ people and guess what? It still does from time to time even with a human piloting the "beast."

It is not impossible to have fully automated planes. We just cannot afford to make any mistakes otherwise people will die so we keep pilots as failsafes.

Nobody will die if your claude code hallucinates.

YMMV but an LLM can hullicinate bug in programmed medical devices which may kill people in unexpected ways

Compliance policies are very strict for industries like aviation and medical devices. You cannot just throw Claude at it legally speaking

There is a product called Microsoft Copilot...

a slightly different metaphor. Copilot suggests it is next to you, helping you pilot... something else. The computer? The system? But "piloting the LLM" changes the relationship. The LLM is the thing that is being piloted.

I think their metaphor is apt. Microsoft Copilot is, in modern terms, the "harness" that "co-pilots" the LLM it manages sessions for, with you as the "pilot". So you are still piloting the LLM.

At least that's how it reads to me.

I thought it was, you're the pilot, copilot is your helper.

it doesn't much matter what you and I think about LLM's quality and output

it only matters what upper management think, and its clear that more and more companies value "good enough" and reducing costs over "good"

a year ago I would have agreed, but the gap is getting smaller all the time... these things can do 90% of the work, and how many people does a company really need for the remaining 10%? certainly not as many as they needed before

The things can do 90% of the work ... but only if used by the right people.

I've seen first hand what less experienced developers produce using the same models, your 90% accuracy suddenly drops to 50%...

With opus 4.8 we're frankly aproaching the 100% of the work, but only if tasked by the right people. A decade ago I worked as an enterprise architect and left it because I preffered coding. Now I'm an enterprise architect again, and we're at the point where I've setup a Microsoft Fabric and integrated a ADLS Gen2 with a Lakehouse building Dimension and Fact tables for our Business Intelligence people with Cowork. A month ago I didn't know what Dimension and Fact tables were in a datawarehouse and now I've not only setup a flow for it I've made it more accurate than what they had before because I understood how BC365 worked and the previous consultants didn't.

We had a PoC in place to get fabric, it had like 500 hours allocated for what I did in a week with cowork, and my product is actually on secure vnet network with Azure identity security with both a test and a production environment delivering actual data.

Cowork even made the damn powerpoint slideshows for decision makers.

The single saving grace right now is that it apparently isn't easy for everyone to do this yet. But I didn't use a whole lot of my knowledge on software engineering to make any of it happen, not even the pandas and arrow code that moves the data behind the scenes. I mainly used my knowledge of NIS2 compliance and general data architecture in a step-by-step process. To me anyone with common sense should be able of doing this, and I really don't think I'm special... but then I teach other people AI at our company and they can barely get it to create a running program. Which is fine for now, but I have to work another 20ish years before I retire, and by then a lot of young people will have grown up with AI, and like I said, I'm not special. I think the only thing that differentes me is that I mash the buttons until it works but also have decades of security and compliance hammered into me.

similar experience here, I don't think I've run into a challenging problem that I've had to solve directly for months

I've been a developer for 25 years

For me it's mostly dealing with humans and bureaucracy now that takes the most time. Actually kicking off an LLM will often (though not always) get me in the ballpark of the right solution, and then iterating with the LLM from there gets me the rest of the way.

I kinda agree I mean almost no one writes code by hand anymore but it doesn't mean we don't contribute any value anymore. I wouldn't exchange the entire r&d department with Claude yet - would you ?

Sure... and people keep finding exceptions like this, but that's not what most developers are doing. We're talking like top 25% has some security and no one else. That's an economy-level change.

sure but even if it's one talented dev doing work that 2 would have done before... where's that work going to go? I don't think we need twice the amount of software

> is regularly wrong, frequently myopic, and just outright dumb constantly.

I give LLMs snippets of text messages exchanges with my wife and I can't believe how dumb the LLMs are of getting basic facts right let alone nuance.

I'm 100% not one those "LLMs are just stochastic parrots" people, but coding and coding-like activities are extraordinarily well fit for LLMs, but for things that there's less training data, LLMs probably do a lot worse

In software dev, for a big finance corp

I like your comment, want to try to expand on it

Comment long but there is a TL;DR at the bottom

My theory is that there are 4 areas to domain knowledge worth taking about here - there may be more but I like 2*2 matrices

1) explicit internal requirements - core of how the how the app should work towards achieving your business objectives - code expresses what should be done and to a pretty large extent, why it should be done - from business unit requirements - we are building a tool to do “X”

2) implicit internal requirements - core of how the how the app should work towards achieving your business parameters and constraints

eg profit = selling price - ( total of costs )

  - code expresses what should be done but really can’t express why. At best it is in the comments

eg if market is EU then tax = 30% (or some value for a table), AI can see what is being done but rationale is not explicit

3) implicit internal requirements - core of how the how the app should work towards achieving your business constraints - code expresses what should be done but really can’t express why. At best it is in the comments

eg if item is “rocket” , shipping = $1m ( we only make rockets in Antarctica and shipping from there is $1m)

4) implicit external requirements - core of how the how the app should work towards achieving your business constraints - code expresses what should be done but really can’t express why. At best it is in the comments

eg if item is “rocket” , add a 3 month gating stage to get approval from government to sell the item and do not collect payment till gating approved - AI can see the code but has no idea why it has to be done

These come from partners, regulation, compliance, auditability etc

So, my theory

AI can be good at the explicit stuff trivially (1, 2) but cannot be good at the implicit stuff (3,4)

It might be able to figure out implicit stuff needs to be done but will probably not be able to figure out why it needs to be done and it will definitely not be able to definitively figure out edge cases for when to do it / not do it

As long as you focus on implicit stuff, you will be fine for a little bit

TL;DR - become good and keep being good at being the person who understands the implicit external drivers of software dev

Lmao please do not say you "pilot" what is a glorified ELIZA. You're a joke that people will have forgotten about in 2 years.

Reply on news.ycombinator.com