Hacker News Reader

How do LLMs work? Yes there's a model and there's inference, but dig deeper. How can you, today, with < $50, train a small GPT-2 class model, how do you fine tune it, what training data do you give it, how do you give it training data, what algorithms do you use for reinforcement learning, what even is RLHF? You don't have to know any of those details to use ChatGPT, just like you don't need to know how a D flip-flop works in a CPU to write some code that runs on one, but if you're looking for feeling like you know what's going on, at their heart, LLMs are still next word prediction engines, so knowing, deeply, how that engine is created and works, would be my idea of mastery of it. Karpathy has a good set of tutorials on this.

>How do LLMs work? Yes there's a model and there's inference, but dig deeper. How can you, today, with < $50, train a small GPT-2 class model, how do you fine tune it, what training data do you give it, how do you give it training data, what algorithms do you use for reinforcement learning, what even is RLHF? You don't have to know any of those details to use ChatGPT, just like you don't need to know how a D flip-flop works in a CPU to write some code that runs on one, but if you're looking for feeling like you know what's going on, at their heart, LLMs are still next word prediction engines, so knowing, deeply, how that engine is created and works, would be my idea of mastery of it. Karpathy has a good set of tutorials on this.

Knowing how an LLM is built, is trained, inferences, isn't "mastery" in the sense that crafts people use the word.

Mastery to a craftsmen is as much a product of mechanical performance as it is knowledge. For a software craftsmen, that means knowing how to code, debug, and architect a software application by hand.

This, plus since you can't realistically run these LLMs at home, the motivation to get invested in them is not really there (for me at least)

I mean, yes I do find the mathematical background of LLMs fascinating, but it's not going to be something that I can put to practical use (much), if that makes sense?

When I would code by hand I often found I was concerned with the house of cards in my head, am I remembering the right variable names, did I forget something that would lead to an exploit, etc. Now I can direct the LLM to do the work and the skill is split between having an understanding of the interactions of the technologies, choosing tech that will be less of a pain to update/migrate. You can even put some LLM-bow grease into finding more efficient approaches catching corner cases etc. And just by asking I can have test frameworks setup, I can drill into some pet peeve and it takes less time. Now we still have lots of ops moments, like the radicel distributed git forge forgetting to use encryption for private repos, and I run into this class of problem in my own casual LLM use, but that's where our taste and attention to detail powers need to be spent in this modern developer world.

Thanks for your perspective

I think one thing that troubles me is that I so far have gotten quite far by 'caring about the little things'.

Now we live in a world where this feels, for the most part, unsustainable to keep up with. When directing LLMs to code something up, it's still occasional useful to have the knowledge and experience of those little things earned over years, but at the same time 1) it seems like I run into fewer opportunities to add to that experience and 2) it seems like you can often just prompt LLMs to 'grill' a design a few times, it will probably surface some of the most important tradeoffs

Yes, the grilling part is where the LLM energy drain is for me, I haven't released any of the software I've been working on yet because they lack polish in my opinion. They may have nobel goals, but no point unleashing more slop on the world. Often when I ask for a correction something new breaks, or it reveals the next corner the model and I didn't think about. I've seen the models come a long way in a short time. I'm tinkering and paying by the token for deepseek flash picked for its economy and using it to teach myself and form opinions on languages and frameworks that are new to me. It's hard to describe the worth of these opinions, but I think they're important, kinda like why I read a lot of HN looking for morsels of sentiment around the tech industry as represented here.

It's funny though, llms are quite terrible at most things really.

They can build things, but it's all terrible really. Their "ideas" are terrible.

Well, I'm somewhat minded to agree with you, but they are fast enough, cheap enough and don't get bored of tedious work, which makes them passable at many things. Maybe.

I feel caught between two worlds, I know people both very 'for' and very 'against'. The business is very keen for people to use them to go faster.

For non-trivial changes, output so far does not meet my quality standard and I tend to spend a lot of time cleaning up after them, probably more than I would have spent doing the work myself. This being much more boring, I hasten to add.

Their comments suck so I feel like I have to go in and rewrite them all.

Maybe I should just let go of the quality standard and accept that in exchange for steaming ahead at full speed, but I'm reluctant (and not just for me, there are also coworkers who I feel like I would be letting down).

I've always been either entirely self-directed, choosing my own goals, or almost entirely self-directed, being given high-level goals, and getting to choose my own means to accomplish them. So my opinions are probably different from somebody who was a smaller part in a chain.

You say that you liked learning how things worked inside out, whether it was Postgres or whatever. You can still do that. You can learn it at a high level, and then it becomes like a magic word in your arsenal. You don't have to know or remember the details of implementation, but only the nature of its costs, benefits, and synergies with other systems. You just have to remember that high-level abstraction, maybe just a single vocabulary word, of a low-level reality - the exact syntax is really no longer a concern.

The better the agents get, the less there is playing manager. Really your job now is to have and execute vision. Execution is largely a function of having a map of the high level vision and low-level realities in your head, or just having in-the-moment intuition about these things... then just speaking it.

Also, there is developing personal libraries. You may find that in your business you repeatedly tackle similar problems. You develop and refine a library that your future agents can utilize. You develop prompts that make your work more automatic and more robust. And when I say you, I mean, you direct your agents to do it.

It's not so different from what it always was, except that years get compressed into months. What was slow and tedious is now... less so.

I don't know how much advantage I have in this new world, having programmed (sometimes more, sometimes less) for 30 years. My brother has never written a line of code in his life. He's been heavy into AI for a while, and he has developed some really incredible stuff. I think he may have already outpaced me. His $1500/mo+ in subscriptions definitely help.