Hacker News Reader

Comment by byzantinegene | Hacker News Reader

you mentioned a very good point about scalability. we're seeing alot of productivity gains, but only from SWEs, which are but a very small segment of the global economy. all other economic use cases require thorough last-mile development and iteration that is not too different with current automation tools.

View on HN

A friend who is a psychologist was telling me he thinks in another year or two insurance companies will insist people see an AI therapist first before being willing to pay for a real person.

LOL, the same AI that has landed companies in court defending themselves against wrongful death lawsuits for helping someone convince themselves suicide is the right answer, and even encouraging them? That AI? I am unclear that any insurance company is going to want a piece of that action anytime soon.

What you've just told me is that psychologists, just like SWEs, are prone to thinking they know how business works but in fact know fuck all.

I'm not sure - like I said, it's just a friend's speculation, but when you consider how far AI has come in the past four years it's not hard to believe that it will simply continue to improve. It seems to already be having a significant impact on the whole software engineering field.

But software engineering is almost certainly the best possible application for LLMs, because there is an immense amount of training data. What training does it have for therapy? Reddit? Egads ;-)

Personally I think we are at the top of the S-curve for this technology. And I suspect a lot of what we see as major recent advances are attributable to improved harnesses as much as any improvement in the abilities of the models. My guess is we will see only minor improvements going forward in the frontier models (barring a major new discovery) and major improvements in smaller models. IMO that is where the real impact will be felt day-to-day, when we can run fairly powerful LLMs in all sorts of places without having to make API calls to huge GPU clusters in the cloud.

Tracks. This is that same speculation that AI will be good enough. Wonder what a crash in that isecase will look like? Increased suicide rates? More instances of psychosis? It might not wven be directly measurable or easily traceable to AI therapists. Would suck

All those automaton tools will eventually be initially one-shotted and then monitored by LLMs though. There probably won't be a "last mile" per se; just constant tweaking and optimizations throughout, within a feedback loop.

What your describing is iterating in the last mile. Your assuming that the AI will iterate in the last mile with the same efficacy that it iterates before that. I think that will fail. Thats the bitter lesson, that adhoc solutions to the last mile (rather than generalozed solutions that scale with training data) asymtotally stall and so dont scale.

Meaning claude code wont be able to make a "claude video editing" or "claude accounting" with the current tech. Human experts will need to encode their knowledge into it for the last mile and that wont scale the way these speculations expect

We aren't seeing productivity gains in software either. What we are seeing is a lot of people who claim to be more productive, but in fact are building piles of tech debt that will fall over before long. But hey, they're building that tech debt really fast!

> but in fact are building piles of tech debt that will fall over before long

This is speculation as well. Its well founded but speculation nonetheless. Youre speculating things will stay the way they have till now.

I do see your point but what makes me consoder the other side is that ive been building an app that reaches ~10k LOC, purely with opus, no code review at all, and it hasnt hit any tech debt issues that i havent easily been able to address. Setting up good context management meant that claude could just figure things out itself.

And for reference this is an app that manages an ethernet camera, runs vehicle detection on the stream, and surfaces the detections on an ipad for operators to inspect and annotate for cellphone usage, so not trivial. Needed good architechtong and design from my end, but it was honestly scarily easy. So idk what the threshold for tech debt crash is but it wasnt there

Reply on news.ycombinator.com