Hacker News Reader

Comment by bigiain | Hacker News Reader

And next month you'll need to add on "Claude Database Pro" or you'll just get a working (for demo purposes with dozens of db rows) but completely un indexed database schema and a refusal to optimise SQL requests.

And the month after you'll need "Claude DataScience Pro" to get any Python Pandas or NumPy code generated.

And and and...

View on HN

While this is a perfectly reasonable thing to expect when the models are competent enough, half the conversation on places like Hacker News are about all the times an LLM has produced garbage that was harmful to a business either by hallucinations, by deleting something critical during the work, or by hitting some endpoint way too often and denial-of-servicing it.

Right now, the software guardrails in LLMs are useful for the same kinds of reasons factories have hardware guardrails: to reduce the rate at which errors become "incidents".

Just because they sometimes delete the production database rather than sometimes spilling a thousand tons of incandescent molten metal over a factory floor, doesn't mean LLMs are safe enough to be used the way they're actually being used.

https://simonwillison.net/2025/Dec/10/normalization-of-devia...

I think you're assuming too much care. Right now they haven't adopted that business model because they don't see it as a viable business model. As soon as they realize that they can lock certain categories of query behind a different subscription they will do that. We saw the same thing with streaming services and basically every other kind of online service -- small, singular subscription followed by a gold rush and then suddenly there's an upcharge for access to every other publisher's catalog of movies.

That kind of thing is basically why I wrote the opening clause of the first sentence.

i.e., yeah, probably.

This is why I'm thankful for Chinese LLM research. They'll keep us honest.

Well, I'd prefer bugfixes over exploit vectors. This will keep us honest.

With pi or better omp it would be incredibly easy to adjust the Claude system prompt so it will be easy to do what the Chinese models or gpt did. That's how the Chinese were training their models btw

Same thing with the weird push towards humanoid robots.

"They can do anything!"

Sure, once you subscribe to the $15/mo laundry package, the $25/mo lawn care package (with the $10/mo hedge trimmer upgrade), and the $10/mo dog-walking package.

And in the end the big reveal is, it was a dude in VR all along, piloting the dumb things remotely. Every single time, without exception.

I think it’s just riding off LLM coattails.

We don’t have good world models. We have had bipedal robotics in various POC demo-ready forms for decades.

It turns out that industrial, purpose build robotics is an easier and better market.

I’m still not completely convinced a robot that’s shaped like a human is the best design other than for PR.

I remember nearly losing my mind at that stupid conveyor belt sorting demonstation because

1. The human beat the robot, but more importantly

2. We've had non-humanoid conveyor belt sorting machinery for decades that beats both

When we are stabbed to death by impoverished dudes who are piloting a robot worth more than a decade of their income to do household chores for 16 hours a day, we will deserve it.

Isn't this inline with trying to leave no money on the table?

I'd hate it, sure, but it wouldn't surprise me.

This is an incredibly unlikely scenario

Reply on news.ycombinator.com