Hacker News Reader

Comment by embedding-shape | Hacker News Reader

> so without LLMs you’re now doing only beginner stuff. You won’t seem like a real dev anymore.

I think it's the opposite. When I'm on the edges of Real Difficult Stuff (TM), then LLMs are less of help compared to when I just need basic web applications or similar "beginner stuff". If what you're working on doesn't exists in the training data at all, the LLMs can sometimes be more in the way than helpful, as they don't seem to even "grasp" the problems correctly.

I still get tons of productivity boosts from LLMs, don't get me wrong, and I wouldn't stop working with them even though they aren't perfect for everything, they do with a lot of monotonous tasks, like all the stuff you outlined. But the moments I'm working without LLMs, that's the time I'm working very much on "not beginner stuff".

View on HN

This is my experience as well. It gets worse when the LLMs have strong priors in subject matter adjacent to your novel work. The generated output will incorrectly pull towards things the LLM has seen in it’s training data.

This is absolutely an issue, and I've run into it several times myself, because I am doing Real Difficult Stuff that I don't think anyone has ever done quite like this before. But I'm leaning heavily on Claude for it. Mostly for the stuff that has already been done before but I need to redo in order to be able to improve on it (mostly various aspects of graph layout algorithms of for the Sugiyama framework), but it's also been surprisingly helpful at helping me write some of the complex new stuff. Those parts require a lot more direct guidance and careful review, but it's still helping me keep a firm grasp on the whole thing. Which I also find surprising, because in the past I've experienced many times that I was losing grasp on a project because I let LLMs do too much. But this time it's the opposite.

The critical IP that isn’t in the weights yet, is often a few hundred to a few thousand lines of code. All the rest, surrounding code, is infrastructure that is indeed in the weights.

yea so many times i have asked my self if i am actually saving time on the actual hard stuff.

What is this real difficult stuff you speak of that only you can do and LLMs can not?

I don't have a concrete example, but it's probably the stuff that starts taking longer to describe in English than to code.

"Make me this thing there are a million examples of" vs. "make me this unique thing in this exact, particular way".

I don't think there's anything an LLM can't write given enough direction. (It's a one-sentence fuck-it one-off on one end of the spectrum, and it degenerately ends with you giving it exact code and prompting it to echo it back to you on the other.)

But the jobs of making basic apps and web pages are the new WordPress jobs in terms of pay, I'd agree.

Write a decent comment, for one. It struggles with some of the firmware stuff I do, though that's more due to a lack of a proper automated feedback loop.

Yeah, it's generally the harness around the model that makes it useful.

Two recent examples I tried to get Sol/Opus to get help with, were they gave more misdirections than beinghelpful: Tracking down the root cause of a synchronization bug in a distributed platform using CRDTs, especially dumb around sequence CRDTs. Also had less fun using Sol/Opus for building and troubleshooting fractures and constraint networks in Houdini.

I'm sure these will be much better as soon as enough data is in the training set, as always. GPT models used to be horrible at electrical stuff as well, today Astra seems pretty OK at it. It changes as they keep extending what use cases are covered by training data and what they test for and so on.

Reply on news.ycombinator.com