Ask HN: Go deep into AI/LLMs or just use them as tools? | Hacker News Reader

Ask HN: Go deep into AI/LLMs or just use them as tools?

195 pointspella_may133 comments

I'm a software engineer with a solid full-stack background and web development. With all the noise around LLMs and AI, I’m undecided between two paths:

1. Invest time in learning the internals of AI/LLMs, maybe even switching fields and working on them

2. Continue focusing on what I’m good at, like building polished web apps and treat AI as just another tool in my toolbox

I’m mostly trying to cut through the hype. Is this another bubble that might burst or consolidate into fewer jobs long-term? Or is it a shift that’s worth betting a pivot on?

Curious how others are approaching this—especially folks who’ve made a similar decision recently.

View on HN

My 2 centes:

1. Learn basic NNs at a simple level, build from scratch (no frameworks) a feed forward neural network with back propagation to train against MNIST or something as simple. Understand every part of it. Just use your favorite programming language.

2. Learn (without having to implement with the code, or to understand the finer parts of the implementations) how the NN architectures work and why they work. What is an encoder-decoder? Why the first part produces an embedding? How a transformer works? What are the logits in the output of an LLM, and how sampling works? Why is attention of quadratic? What is Reinforcement Learning, Resnets, how do they work? Basically: you need a solid qualitative understanding of all that.

3. Learn the higher level layer, both from the POV of the open source models, so how to interface to llama.cpp / ollama / ..., how to set the context window, what is quantization and how it will affect performances/quality of output, and also, how to use popular provider APIs like DeepSeek, OpenAI, Anthropic, ... and what model is good for what.

4. Learn prompt engineering techniques that influence the qualtily of the output when using LLMs programmatically (as a bag of algorithms). This takes patience and practice.

5. Learn how to use AI effectively for coding. This is absolutely non-trivial, and a lot of good programmers are terrible LLMs users (and end believing LLMs are not useful for coding).

6. Don't get trapped into the idea that the news of the day (RAG, MCP, ...) is what you should spend all your energy. This is just some useful technology surrounded by a lot of hype of all the people that want to get rich with AI and understand they can't compete with the LLMs themselves. So they pump the part that can be kinda "productized". Never forget that the product is the neural network itself, for the most part.

This, 100%. A full-stack engineer will likely have at least a solid understanding of the HTTP protocol, HTTPS, WebSockets, the interface layer between the frontend server and their chosen Web webdev stack, and so on. Then a more general understanding of networking protocols, TCP vs UDP, DNS, routing, etc. In general, you need to have a solid understanding of the layer below where you're working, some understanding of the layer below that, and so on, less and less detail needed for each layer down.

(That's not to say that you shouldn't bother with learning more -- more knowledge is always good -- or that the OP specifically only knows that. It's more a sensible minimum.)

My own "curriculum" for that has been Jeremy Howard's Fast AI course and Sebastian Raschka's book "build an LLM from scratch". Still working through it, but once I'm done I think I'll be solid on your point 2 above. My guess is that I'll want to learn more, but that's out of interest more than because I think its necessary.

As someone who I both respect a lot and know is really knowledgeable about the latest with AI and LLMs: can you clarify one thing for me? Are all these points based on preparing for a future where LLMs are even better? Or do you think they're good enough now that they will transform the way software is built and software engineers work, with just better tooling?

I've tried to keep up with them somewhat, and dabble with Claude Code and have personal subscriptions to Gemini and ChatGPT as well. They're impressive and almost magical, but I can't help but feel they're not quite there yet. My company is making a big AI push, as are so many companies, and it feels like no one wants to be "left behind" when they "really take off". Or is that people think what we have is already enough for the revolution?

I think that LLMs already changed the way we code, mostly, but I believe that agentic coding (vibe coding) is right now able to produce only bad results, and that the better approach is to use LLMs only to augment the programmer work (however it should be noted that I'm all for vibe coding for people that can't code, or that can't find the right motivation. I just believe that the excellence in the field is human+LLM). So failing to learn LLMs right now is yet not catastrophic, but creates a disadvantage because certain things become more explorable / faster with the help of 200 yet-not-so-smart PHDs in all the human disciplines. However other than that, there is the fact that this is the biggest technology emerging to date, so I can't find a good reason for not learning it.

> Learn how to use AI effectively for coding. This is absolutely non-trivial, and a lot of good programmers are terrible LLMs users (and end believing LLMs are not useful for coding).

I've been asking this on every AI coding thread. Are there good youtube videos of ppl using AI on complex codebases. I see tons of build tic-tac-to in 5 minutes type videos but not on bigger established codebases.

You may want to check my channel perhaps. There are videos of coding with LLMs doing real world things. Just search for "Salvatore Sanfilippo" on YouTube. The videos about coding+LLM are mostly in English.

IIRC the guy who makes Aider (Paul Gauthier) has some videos along these lines, of him working on Aider while using Aider (how meta).

My problem with 5. is that there are many unknowns, especially when it comes to agents. They have wildly different system prompts that are optimized on a daily basis. I’ve noticed that Gemini 2.5 Pro seems way dumber when used in the Copilot agent, vs me just running all the required context through OpenRouter in Continue.dev. The former doesn’t produce usable iOS tests, while the latter was almost perfect. On the surface, it looks like those should be doing the same thing; but internally, it seems that they are not. And I guess that means I should just use Continue, but they broke something and my workflow doesn’t work anymore.

And people keep saying you need to make a plan first, and then let the agent implement it. Well I did, and had a few markdown files that described the task well. But Copilot‘s Agent didn’t manage to write this Swift code in a way that actually works - everything was subtly off and wrong, and untangling would have taken longer than rewriting it.

Is Copilot just bad, and I need to use Claude Code and/or Cursor?

> Is Copilot just bad, and I need to use Claude Code and/or Cursor?

I haven't used Claude Code much, so cannot really speak of it. But Copilot and Cursor tends to make me waste more time than I get out of it. Aider running locally with a mix-and-match of models depending on the problem (lots of DeepSeek Reasoner/Chat since it's so cheap), and Codex, are both miles ahead of at least Copilot and Cursor.

Also, most of these things seems to run with temperature>0.0, so doing multiple runs, even better with multiple different models, tend to give you better results. My own homegrow agent that runs Aider multiple times with a combination of models tend to give me a list of things to chose between, then I either straight up merge the best one, or iterate on the best one sometimes inspired by the others.

I never ever use agents for coding. Just the web interface of Gemini, Claude, ..., you are perfectly right that agentic coding just creates a layer of indetermination and chaos.

Agreed with most of this except the last point. You are never going to make a foundational model, although you may contribute to one. Those foundational models are the product, yes, but if I could use an analogy: foundational models are like the state of the art 3D renderers in games. You still need to build the game. Some 3D renderers are used/licensed for many games.

Even the basic chat UI is a structure built around a foundational model; the model itself has no capability to maintain a chat thread. The model takes context and outputs a response, every time.

For more complex processes, you need to carefully curate what context to give the model and when. There are many applications where you can say "oh, chatgpt can analyze your business data and tell you how to optimize different processes", but good luck actually doing that. That requires complex prompts and sequences of LLM calls (or other ML models), mixed with well-defined tools that enable the AI to return a useful result.

This forms the basis of AI engineering - which is different from developing AI models - and this is what most software engineers will be doing in the next 5-10 years. This isn't some kind of hype that will die down as soon as the money gets spent, a la crypto. People will create agents that automate many processes, even within software development itself. This kind of utility is a no-brainer for anyone running a business, and hits deeply in consumer markets as well. Much of what OpenAI is currently working on is building agents around their own models to break into consumer markets.

I recommend anyone interested in this to read this book: https://www.amazon.com/AI-Engineering-Building-Applications-...

I agree that instrumenting the model is useful in many contexts, but I don't believe it is something so unique to value Cursor such valuation, or all the attention RAG, memory, MCP get. If people say LLMs are going to be commodities (we will see) imagine the layer about RAG, tool usage, memory...

The progresses we are seeing in agents are 99% due to new LLMs being semantically more powerful.

Thanks for this breakdown, I guess I'm largely in the window of points 3-6.

Any suggestion on where to start with point 1? (Also a SWE).

Thank you for sharing. Do you recommend any courses or books for following that path?

For SWEs interested in "AI Engineering" (either getting involved in how models work, or building applications on them), there's a critical paradigm shift in that using "AI" requires more of an experimental mindset than software engineering typically does.

- I strongly recommend Chip Huyen's books ("Designing Machine Learning Systems" and "AI Engineering") and blog (https://huyenchip.com/blog/).

- Andreessen Horowitz' "AI Cannon" is a good reference listicle (https://a16z.com/ai-canon/)

- "12 factor agents" (https://github.com/humanlayer/12-factor-agents)

I come from a more traditional (PhD) ML/DL background. I wouldn't recommend getting into (1) because the field is incredibly saturated. We have hundreds of new, mostly low quality, papers each day. If you want to get into AI/ML on a more fundamental level now is probably the worst time in terms of competition. There are probably 100x more people in this field than there are jobs, and most of them have a stronger background than you if you are just starting out.

Looks like OP’s curiosity isn’t just about deep diving LLMs —he’s probably itching to dig into adjacent topics like RAG, AI pipelines, and all the other adjacent LLM rabbit holes.

So in that case I don’t see why not?

I just wanted to second the previous comment, and this is even for adjacent fields. Also a PhD AI/ML grad, and so many of us are out of work at the moment that we'll happily settle for prompt engineering roles, let alone RAG etc., just to maintain appearances on CVs/eligibilty for possible future roles.

Kinda surprised of that, actually. Sure, I get that research interest in any if the "traditional" ML methods (SVMs, markov models, decision trees, that kind of stuff) is probably essentially dead right now, but I had thought interest in neural networks and "understanding" what LLMs do internally to be ballooning.

I could imagine that even those "ancient" techniques might some day make a comeback. They're far inferior to LLMs in terms of expressive power, but they also require literally orders of magnitude less memory and computation power. So when the hype dies down, interest in solutions that don't require millions in hardware cost or making your entire business dependent on what Sam Altman and Donald Trump had for breakfast might have a resurgence. Also, interestingly enough, LLMs could even help in this area: Most of those old techniques require an abundance of labeled training data, which was always hard to achieve in practice. However, LLMs are great at either labeling existing data or generating new synthetic data that those systems could train on.

If there indeed were 100x people more than jobs the salaries would tank. And this is not the case at all with AI/ML salaries being much higher than regular devs

> 100x people more than jobs

there aren't 100x 'top shelf' ml engineers.

There aren't a lot of jobs self taught ml programmers like there are for self taught python programmers.

I don't think so, unless you work for Top AI companies or teams in big tech.

Really?

> Is this another bubble that might burst

I see this a lot, but I think it's irrelevant. Even if this is a bubble, and even if (when?) it bursts, the underlying tech is not going anywhere. Just like the last dotcom bubble gave us FAANG+, so will this give us the next letters. Sure, agentsdotcom or flowsdotcom or ragdotcom might fail (likely IMO), but the stack is here to stay, and it's only gonna get better, cheaper, more integrated.

What is becoming increasingly clear, IMO, is that you have to spend some time with this. Prompting an LLM is like the old google-fu. You need to gain experience with it, to make the most out of it. Same with coding stacks. There are plenty of ways to use what's available now, as "tools". Play around, see what they can do for you now, see where it might lead. You don't need to buy into the hype, and some skepticism is warranted, but you shouldn't ignore the entire field either.

This is just survivorship bias speaking. Some bubbles have a useful core somewhere, but that doesn't mean that every (or even most) bubble does.

agree on this point, like web3 and blockchain is not essential as of today (CMIIW).

However not in the case of AI (agentic AI / LLM), because simply they already have a use case, and a valid one. Contextual query and document searching / knowledge digging will be there to stay, either in form of current agentic model or different one.

We already had text search, and I don't see the value in adding a bullshittifier on top of that.

Many people do. I can't imagine ever going back to opening ten tabs for a question. It's not like information on the internet was bullshit-free pre-LLMs. For me, it's easier to verify information than to find it.

And what does your verification process look like then, if it somehow doesn't involve doing any research yourself anyway?

This is the most reasonable stance, and I see a lot of smart people take it. The bubble will burst and some winners will remain. Those companies will make bank because the tools are useful to those that know how to use them effectively will excel.

Depends on what you want to do. But my 2 cents are that like all new technology, LLMs will become a commodity. Which means that everybody uses them but few people are able to develop them from scratch. It's not different from other things like databases, GPU drivers, 3D engines for games, etc. That all involves a lot of hardcore computer science and math. But lots of people use these things without being hindered by such skills.

It probably helps a little to understand some of the internals and math. Just to get a feel for what the limitations are.

But your job as a software engineer is probably to stick things together and bang on them until they work. I sometimes describe what I do as being a glorified plumber. It requires skills but surprisingly few skills related to math and algorithms. That stuff comes in library form mostly.

So, get good at using LLMs and integrating what they do into agentic systems. Figure out APIs, limitations, and learn about different use cases. Because we'll all be doing a lot of work related to that in the next few years.

LLMs already are a commodity. Google has already kicked off the competitive price wars. Plus I’ve already seen some local companies just buy a beefy GPU server and deploy an open LLM model. While OpenAI is still trying to push quality, their competitors have already positioned themselves to offer the lowest possible prices. And since Nvidia has no easy path for scaling up compute anymore, I also wouldn’t bet on much larger LLMs anytime soon.

That means, if you learn more about the internals of LLMs, your market angle is going to be artisanal customised models. Fashion is commoditised, but people still pay for a custom tailored suit. In the same way companies will continue to pay for finetunes optimised for their business usecase.

If you decide to focus more on the application of LLMs, you should really invest into high-level architectural skills. Good “code completion” models can already do what an outsourced 10 bucks per hour developer used to do. Your job in the future is going to be to decide the structure of which fuse and against the towel and or which type of state is being stored and managed. But the actual coding of the UI forms and the glue code to synchronise from an SQL query to the client state, that part is probably going to be fully outsourced to LLMs.

I think this is the key point - LLMs will go through a commoditization phase and I think you left out a key example from a technology and business context: search engines. There was a huge trend where everyone needed search and was building search, etc. and a couple decades later there are lots of companies that evaporated and a few left standing.

There was also a dot com bubble, mostly bursting not because of search but because there were a lot of what today would be "AI startup" but is just a web app calling AI Api's. So there's likely to be some bubble burst but it should be smaller maybe hitting more of these small tools that eventually become features.

>It's not different from other things like databases, GPU drivers, 3D engines for games, etc.

Not quite the same. E.g. databases are a part of the system itself. It's actually pretty helpful for a SWE to understand them reasonably deeply, especially when they're so leaky as an abstraction (arguably, even the more nuanced characteristics of your database of choice will influence the design of your whole application). AI/LLMs are more like dev tooling. You don't really need to know how a text editor, compiler or IDE works.

We have a service at work which categorizes internal documents and logs, then triggers some automation depending on the category. It processes maybe 100 per day. Previously we only used some combination of metadata, regex, and NLP to categorize. Now a call to a LLM is part of that service. We save a lot of manual time where we used to have to resolve unknown documents. The LLM can help fill out missing data, too. It's all stored as annotations so it's clear who/what edited the data.

Granted this is a pretty simple task and a low stakes scenario, but I don't think we should limit ourselves to assuming AI will always only be dev tooling.

That said, I think there is this thing in between of developing LLMs and using LLMs via APIs and the lines are of cause blurry: Training LLMs (or other neural networks). So best I think is to start digging on the surface and going deeper as long as you feel comfortable. Maybe at a certain point you will have the wish for more power full hardware. Thats the point where you need to decide how much to get invested or to rent a cluster.

But the question is what mindset will allow you to put yourself ahead of the rest. Because I suppose the OP doesn't want to end up as just another mediocre programmer.

Do what interests you.

Every programmer really is just mediocre. There is no perfect software yet. Hence people who built it are mediocre.

Like any skillset, programming skills likely form a distribution pattern. There are exceptional programmers out there, I've worked with a few. "Every programmer really is just mediocre" merely indicates you have only worked with mediocre colleagues and are one yourself.

> There is no perfect software yet.

"Software" you refer to is actually 'software product', not merely 'code'. So the reality is that even with exceptional programming talent, the art of making great software products is out of reach of most teams and companies. Vision, management, product development, accurate grasp of the user needs, ..., none of these are "programming" skills.

I even consider well respected devs mediocre. Obviously there is a distribution, like with everything. But even the best of the best produce garbage

I've met one or two great programmers. But perfect software that solves a significant problem can't usually be built by one person, so it's rare.

Well, in any case, llms are certainly not perfect.

Hence why I avoid to use them.

Are your other tools/languages perfect or imperfect?

Imperfect != useless.

There are a lot of paths to become T shaped.

> become T shaped.

my middle manager buzzwords this 26 times a day. triggers me.

Same. Yet being a generalist has always been the most interesting to me so I carried on that path. Ironically, now I can use an LLM for depth, I’m the one being asked how I manage to ship so much. It’s in part due to how I use LLMs for depth whilst relying on my natural breadth.

Having wide shoulders is cool but how does it help with software engineering?

I’d recommend you simply follow your curiosity and not take this choice too seriously. If you’re simply doing this for career purposes, then the honest answer is that absolutely no one knows where these fields will go in the next couple years so I wouldn’t take anyone’s advice too seriously.

But as for my 2 cents, knowing machine learning has been valuable to me, but not anywhere near as valuable as knowing software dev. Machine learning problems are much more rare and often don’t have a high return on investment.

As an MLE I get a decent amount of LinkedIn messages. I think I got on someone’s list or something. I would bucket the companies into two groups:

1) Established companies (meta/google/uber) with lots of data and who want MLEs to make 0.1% improvements because each of those is worth millions.

2) Startups mostly proxying OpenAI calls.

The first group is definitely not hype. Their core business relies on ML and they don’t need hype for that to be true.

For the second group, it depends on the business model. The fact that you can make an API call doesn’t mean anything. What matters is solving a customer problem.

I also (selfishly) believe a lot of the second group will hire folks to train faster and more personalized models once their business models are proven.

I think about this a lot. If you're early in your career, it must feel like you're staring at a technological fork in the road, with AI standing there ominously, waving both paths like it's the final boss in a RPG game.

Between your two options, I’d lean toward continuing to build what you’re good at and using AI as a powerful tool, unless you genuinely feel pulled toward the internals and research side.

I’ve been lucky to build a fun career in IT, where the biggest threats used to be Y2K, the dot-com bubble, and predictions that mobile phones would kill off PCs. (Spoiler: PCs are still here, and so am I.)

The real question is: what are you passionate enough about to dive into with energy and persistence? That’s what will make the learning worth it. Everything else is noise in my opinion.

If I had to start over today, I'd definitely be in the same uncertain position, but I know I'd still just pick a direction and adapt to the challenges that come with it. That’s the nature of the field.

Definitely learn the fundamentals of how these AI tools work (like understanding how AI tools process context or what transformers actually do). But don’t feel like you need to dive head-first into gradient descent to be part of the future. Focus on building real-world solutions, where AI is a tool, not the objective. And if a cheese grater gets the job done, don’t get bogged down reverse-engineering its rotational torque curves. Just grate the cheese and keep cooking.

That’s my 2 cents, shredded, not sliced.

Building AIs has always been there - it's a (fuzzy, continuous to its complement) way to engineer things. Now we have a boom over the development of some technologies (some next-layer NN implementations).

If you are considering whether the future will boost the demand to build AIs (i.e. for clients), we could say: probably so, given regained awareness. It may not be about LLMs - and it should not, at this stage (it can hit reputation - they can hardly be made reliable).

Follow the Classical Artificial Intelligence course, MIT 6.034, from Prof. Patrick Winston - as a first step.

Depends on your goals. :)

If you're good at what you're doing right now and you enjoy it — why change? Some might argue that AI will eventually take your job, but I strongly doubt that.

If you're looking for something new because you are bored, go for it. I tried to wrap my head around the basics of LLMs and how they work under the hood. It’s not that complicated — I managed to understand it, wrote about it, shared it with others, and felt ready to go further in that direction. But the field moves fast. While I grasped the fundamentals, keeping up took a lot of effort. And as a self-taught “expert,” I’d never quite match an experienced data scientist.

So here I am — extensively using AI. It helps me work faster and has broadened my field of operation.

3) go back to school and study something that isn't done entirely on a computer and requires human physical presence (for now). Learning plumbing, electrical wiring, welding, etc. are options. Even if you don't make that your primary path, it never hurts to have a fallback plan JUST IN CASE some of the buzz around AI-fueled job displacement turns out to be valid.

Or, if you believe there may be some merit to "AI is coming for your job" meme, but really don't want to do blue collar / skilled trades work, at least go in with the mindset of "the people who build, operate, and maintain the AI systems will probably stay employed at least a little bit longer than the people don't". And then figure out how to apply that to deciding between one or both of your (1) and (2) options. There may also be some white collar jobs that will be safe longer due to regulatory reasons or whatever. Maybe get your physician's assistant license or something?

And yes, I'm maybe playing "Devil's Advocate" here a little bit. But I will say that I don't consider the idea of a future where AI has meaningful impact on employment for tech professionals to be entirely out of the question, especially as we extend the timeline. Whatever you think of today's AI, consider that it's as bad right now as it will ever be. And ask what it will be like in 1 year. Or 3 years. Or 7 years. Or 10 years. And then try to work out what position you want to be in at those points in the timeline.

Going into trades sounds nice on paper but the salaries there were mostly always low because you need only a handful of those to saturate market needs.

Its not IT where you can create value from thin air and thus grow the market and increase need for even more professionals.

As soon as a tiny percent goes into trades (bet tons of new people already doing this) the market will be oversaturated in a few years when they finish apprenticeships.

After that it will be harder to find a job than in IT with AI around the corner.

Yes, in the worst case scenario we wind up with basically nobody having jobs. I mean, when humanoid robots get sufficient dexterity, they can even come for the skilled trades folks as well, as far as that goes.

Look, I don't know if any of this is actually going to to come to pass or not. But it seems at least a little bit less like pure sci-fi now than it did a decade or two ago.

Anyway, if we play along with the thought experiment of asking "what happens to our society when a very large swathe of the human population is no longer needed to exchange their labor for wages?" it really leads one to wonder what kind of economic system(s) we'll have and if we'll find a way to avoid a straight up dystopian hellscape.

> in the worst case scenario we wind up with basically nobody having jobs

Call me optimistic or whatever, but isn't that the best case scenario? If having a full-time job is basically just for the 0.1% or whatever, then we must have figured out a different way of distributing goods and solving peoples needs, that doesn't involve "trading time for money" (a job), and that sounds like it can be a good thing, not "worst case scenario".

We live in a system the prioritizes profits over everything.

> Call me optimistic or whatever, but isn't that the best case scenario?

The gains of technology are mostly captured by those with capital, not those with labor. Look at wage growth over the last few decades as well as productivity growth to have confirmation.

There’s no reason to believe given the current trend that benefits will be evenly distributed to the 99-97% of wage earners.

> We live in a system the prioritizes profits over everything.

Right, but in this hypothetical future where "basically nobody have a job", would we still live in such a system? If so, where do the money come from if people don't work for it?

Yeah, that's sort of what I was getting at when I said

> it really leads one to wonder what kind of economic system(s) we'll have

It's not clear how a future economy works if nobody has a job. If nobody has a job, how do they acquire food, shelter, clothes, etc? And if people don't have money to buy "stuff" then what need is there for factories to produce "stuff", or stores to sell "stuff" and so on? So does economic output just drop to zero? Or near zero?

Of course there are proposals around UBI and what-not and maybe one or more of those is the answer (if all of this comes to pass in the first place). But it seems to me that there are still a lot of questions to answer.

> Call me optimistic or whatever, but isn't that the best case scenario?

Fair enough. I should have said "worst case, in the context of the system (as it is today) where a job is the primary way we have of gaining income to support ourselves (food, housing, clothing, etc)".

So yeah, if the day comes when nobody has jobs AND we can work out a social/economic system that doesn't leave people destitute and starving in the streets, then it could be the "best case". I'm just not sure how we get there, at the moment.

Or we just have the vast majority of people homeless and starving and helpless against the robot armies.

Again, probably the optimist in my shining through, but hardship seems to (at least where I am) trigger people to react against it, and go out on the street to protests. Clearly, it isn't the case everywhere, but I do believe (or maybe hope) that if we ever get to that point, people won't give up until their life gets at least "Good Enough".

Nobody is expecting you to be able to derive and write automatic differentiation (AD) library from scratch but it's always good to know the fundamentals [1].

Andriy Burkov has written excellent trilogy books series on AI/LLMs namely "The Hundred-Page Machine Learning Book" and "Machine Learning Engineering" and the latest "The Hundred-Page Language Models Book" [2],[3],[4].

Having said that, the capability of providing useful AI/LLMs solutions for intuitive and interactive learning environment, training portal, standards documentation exploration, business and industry rules and regulations checking, etc based on the open-source local-first data repository with AI/LLMs are probably the killer application that're truly useful for end users, for examples here [5],[6].

[1] Automatic differentiation:

https://en.wikipedia.org/wiki/Automatic_differentiation

[2] The Hundred-Page Machine Learning Book:

https://www.themlbook.com/

[3] Machine Learning Engineering:

https://www.mlebook.com/wiki/doku.php

[4] The Hundred-page Language Models Book

https://www.thelmbook.com/

[5] Local-first software: You own your data, in spite of the cloud:

https://www.inkandswitch.com/essay/local-first/

[6] AI-driven chat system designed to support students in the Introduction to Computing course (ECE 120) at UIUC, offering assistance with course content, homework, or troubleshooting common problems. It serves as an educational aid integrated into the course’s learning environment:

https://www.uiuc.chat/ece120/chat

When I was in my postdoc (applied human genetics), my advisor's rule was that you needed to understand the tools you were using at a layer of abstraction below your interface with them.

For example, if we wanted to conduct an analysis with a new piece of software, it wasn't enough to run the software: we needed to be able to explain the theory behind it (basically, to be able to rewrite the tool).

From that standpoint, I think that even if you keep with #2, you might benefit from taking steps to gain the understanding from #1. It will help you understand the models' real advantages and disadvantages to help you decide how to incorporate them in #2.

> my advisor's rule was that you needed to understand the tools you were using at a layer of abstraction below your interface with them.

Very wise advice! And the more complex systems are, the more this is truly needed.

Both are tough.

1/ There aren't many jobs in this space. There are still far more companies (and roles) that need 'full-stack development' than those focused on 'AI/LLM internals.' With low demand for AI internals and a high supply of talent—many people have earned data science certificates in AI hoping to land lucrative jobs at OpenAI, Anthropic, etc.—the bar for accessing these few roles is very high.

2/ The risk here is AI makes everyone good at full-stack. This means more competition for roles, less demand for roles (now 1 in-experienced engineer with AI, can output 1.5x the code an experience Senior engineer could do in 2020).

In the short/medium term, 2/ has the best risk/reward function. But 1/ is more future proof.

Another important question is where are you in your career? If you're 45 years old, I'd encourage you to switch into leadership roles for 2/. This wont be replaced by AI. If you're early in your career, it could make more sense to switch.

From my prespective it's a bubble, very similar to the dot com bubble. All businesses are integrating it into everything, often where it's unnecessary or just confusing.

But I believe that the value will come after the bubble is burst, and the companies which truly create value will survive, same as with webpages after the dot com bubble.

My recommendation would be to use them as a tool to build applications. There's much more potential there, and it will be easier to get started as an engineer.

If you want to switch fields and work on LLM internals/fundamentals in a meaningful way, you'd probably want to become a research scientist at one of the big companies. This is pretty tough because that's almost always gated by a PhD requirement.

AI is a scientific discipline. Software development is an engineering discipline.

Do you like science? Then dive deep into LLMs. Be ready for science, though. It involves shooting a thousand shots in the dark until you discover something new. That's how science gets done. I respect it, but I personally don't love doing it.

Do you like engineering? That's when you approach a problem and can reason about a number of potential solutions, weigh the pros and cons of each, and pick one with the appropriate trade-offs. It's pretty different from science.

This makes frontier research sound very similar to gambling. Any correlation in personality types?

I posted a recent Show HN[1] detailing why I felt the need to understand the basics of what LLMs do, and how they do it. Even though I've no interest in building or directly training LLMs, I've learned the critical importance of preparing documentation for LLM training to try and stop AI models generating garbage code when working with my canvas library.

[1]https://news.ycombinator.com/item?id=44079296

Focussing on the inner workings of them may well end up being a type of programming you don’t enjoy: endless tweaking of parameters and running experiments.

Learning to work with the outputs of them (which is what I do) can be much more rewarding. Building apps based around generative outputs, working with latency and token costs and rate limits as constraints, writing evals as much as you write tests, RAG systems and embeddings etc.

Hi, I am working in making the term "llm developer" more popular in France and train people to this new job. We will need a bunch of them in the months/years to come to implement advanced AI systems after companies manage to properly pick and set up their AI platforms. Currently people would tend to involve data scientists into this job, but data scientists are often less versed into the software engineering aspect, eg when they work more on notebooks than web apps. The job is akin to being a web developer, so a "normal" developer but specialized in a certain field. Knowing the internals of LLMs is a big bonus, but you can start your journey with treating them as black box tools and still craft relevant solutions. You'll need to learn about running systems with databases (vector, graph, relational and nosql are all useful) and plugging multiple services together (docker, kubernetes, cloud hosting).

Isn't this what originally data engineers were supposed to be? I get that the role has probably become "clicks buttons in the Azure GUI", but the "good" ones are basically backend developers that specialize on data stacks.

The data scientist roles have had a similar drift in my experience. They used to be "statistician who can code" or "developer who knows some stats", what we got is "clicks buttons in the Azure GUI".

I guess for the "backend" part it may be true, but there is a new "frontend" skillset of being able to actually implement a complex agentic workflow, and put that into a functioning product, which can be a lot of code. So closer to web dev in the end. I have nothing against button clicking if it does work, as it may be a sign of maturity for a technical field. I click on a button in my car and turn a wheel and it usually go where I want, feels great. The equivalent would be the Microsoft ecosystem here.

3. Focus on leveraging AI to solve real world problems.

You don't need to deep dive into the maths. You'll need to understand the limitations, the performance bottlenecks, etc. RAGs, Vector DBs, etc

IMO, you're a woodworker, a craftsman that builds solid products. You've been using a hacksaw and hammer all these years, now someone invented a circular saw and drill and people can move a lot faster. And now even relatively previously inept people are able to do woodwork.

Do you need to understand how the circular saw and drill are made?

To continue with your analogy: maybe they don't need understand every detail, but they should know how they function, what safety precautions to take, and when it is a better/more useful tool compared to what they're currently using.

That doesn't mean knowing every single bit there is to know about it, but a basic understanding will go a long way in correctly using it.

I believe you should do what you genuinely find interesting. Go for 1, dig into internals, read some papers, and see how it goes. Even if you decide not to get into ML/AI, learning how stuff works is always rewarding.

Lots of good answers here about part of your question: is it hype or not? In that it likely isn't going away, and is becoming a valid force multiplier, it's not.

However, this question is better answered by asking yourself what you're interested in. Do you _want_ a deeper understanding of AI/ML? If so, jump in. If you're not genuinely interested it'll be an interminable slog, and you'll revert to doing whatever you actually want to do eventually.

Nothing wrong with continuing to develop web/full stack apps while leveraging the new tools; that's also quite interesting.

When learning new stuff I consider 2 things: - How much fun is the actual learning? - Can I actually apply what I am learning?

So I would learn things that are either fun for you to learn or things that you can directly apply.

For AI this means you probably should learn about it if you are really interested and enjoy going through build-your-own-NN tutorials or if you have good chances of switching to a role where you can use your new skills.

Edit: Basically investing anything (also time) is risky. So invest in things that directly pay off or bring you joy regardless of the outcome.

Maybe I'm too late to the party, but here are my 2 cents.

AI/LLMs for web dev is like StackOverflow on steroids. Was it worth learning the model of SO site, or Google, in order to become a good dev? I guess, no.

LLMs jobs in this analogy are like becoming search engine developer. Completely different area, and different career, I guess.

To upgrade a CV and just skills, I would suggest learning another backend language, like Go or Rust, if they're not on your CV already. I chose Rust, and it boosted my understanding of what's happening in code. You may also choose Go, and offer speed boosts for Python/JS backend code, as many employers are more willing to use it than Rust. I hesitated to do that, and turned out, I was wrong, and I could have offered impressive speed boosts at one of my jobs, where background jobs would work for hours.

This can be any tech that is near, but feels like beyond a barrier, and people hesitate trying it.

As for other jobs - take a look at college/uni teaching: it brings status (if you do it more, even tangible titles), and boosts social network, which will yield crops in 5-15 years.

Another thing is something that requires certification. This is what people hesitate to try or fail at tests. Jobs protected by certificates are supposedly more secure. And learning as an adult is much cooler, because now you've seen yourself learn stuff on your own and can apply methods to the process.

It's your choice, but it's definitely not ,,just another tool''.

Most of my LLMs made lots of mistakes, but Codex with $200 subscription changed my workflow totally, and now I'm having 40 pull requests/day merged.

Treat LLMs as interns, increase your test coverage with them to the point that they can't ruin your codebase and get really good at reviewing code and splitting tasks up to smaller digestible ones, and promote yourself as team leader.

What kind of tasks you give Codex?

I gave it an honest chance, but couldn’t get a single PR out of it. It would just continue to make mistakes. And even when it got close I asked it a minor tweak and it made things worse. I iterated 7 times on the same small problem.

> What kind of tasks you give Codex?

Currently in the stage of evaluating Codex (mostly comparing it to Aider and my own homegrown LLM setup). I'm able to get changes out of it, that mostly make sense, but you really need to take whatever personal guidelines you have for coding and "encode" them into the AGENTS.md, and really focus on asking the right question/request changes in the right way.

Without AGENTS.md, it seems to go of the wrong end really quickly, and end up with subpar code. But with a little bit of guidance, I do get some results at least. This is the current AGENTS.md I'm using for some smaller projects: https://gist.github.com/victorb/1fe62fe7b80a64fc5b446f82d313...

With that said, it does get mislead sometimes, and the UX isn't great for the web version. It's really slow, you can't customize the environment, the UI seems to load data in a really weird way leading to slowdowns and high latencies, and overall it's just cumbersome. My homegrown version is way faster for the iterations, + has stateful PRs it can iterate on and receive line comment feedback on, but the local models I'm using are obviously worse than the OpenAI ones, so I'd still say Codex is probably overall better, sadly.

You can review and approve 40 PR's a day from intern quality work?

Sure, LGTMx40 and call it a day .What could possibly go wrong.

I'm sceptical of 40 PRs per day.

In an 8 hour workday you are merging one new PR every 12 minutes?

I'm very sceptical that anyone can review a significant chunk of code that fast, unless these are all one and two liners that pass review on the first go.

In this best case scenario, where no review results in reworking the PR, and you can review and merge every 12 minutes, without any breaks of any sort, why is your review even required?

It's not 8 hour work day, I just did it all day long from waking up to sleeping. And of course it contains lots of corrections. sometimes it's 30 lines with some bug that I see, then I ask codex to fix the bug in a separate pull request. It's an open source web framework I have been working on ( https://github.com/adamritter/pageql).

To tell you the truth it slowed down yesterday as most of the features were done and I had to fix memory/signal/callback leaks, but even that was done much faster than if I did it by hand.

I had done coding on my phone while buying flowers in a gardening store, in my bed waking up without getting out my laptop etc... so yeah, it's not for those people who take the 8 hour work day strictly.

Ia this a personal project or prod startup with clients?

Personal

Can you share your source?

Sure, it's an open source reactive web framework, the pull requests are public. I want to announce it here on HN soon, I just still have a few serious bugs to fix:

https://github.com/adamritter/pageql

Awesome. There is a lot of discussion around coding agents but I don’t find a lot of real world examples.

This will be interesting to look at, thanks for sharing!

LLMs are part of soft-computing, i.e. contrary to traditional (algortihm-based) computing sometimes things won't get the right result (or, just as bad, the right result in the wrong format). Engineering solutions with LLM is a lot of fiddling, which is experimental rather than analytical/logical.

It is worth getting use to that mind-set, and then use LLMs as a tool (they are likely here to stay, because big tech have started to integrate features based on them everywhere, for better or worse). So this is your option (2.). Personally, I prefer software I use NOT to be smart, but to be 100% deterministic.

But already my favorite LaTeX authoring environment (Overleaf) has a button pop up called "fix this" that auto-resolves syntax errors, many of which overwhelm my doctoral students that no longer read books end-to-end (here, to learn LaTeX).

Gradually, you may dive deeper into the "how", driven by either need or curiosity, so eventually you will probably have done both (2.) and (1.). - in the same way that you will have first learned SQL before learning how replication, transactions, data buffers and caches, or query optimizers are implemented.

I don't really understand why would you even consider going deep if this is not something you have experience with or strong interest. Sure, it is good to know how some of the things work under the good, but you can be perfectly capable developer by reading docs and knowing how to use tools - without needing to know how to write them or how they do complex things under the hood. Take databases as example, network stack, etc.

Just because a field is popular, doesn't mean you should switch to going deep into it. But that doesn't mean you shouldn't use it - it costs a few dollars to try it out and see whether it fits your workflow. If it does, this is great and you can be more productive and focus on stuff that you can solve and LLM can't, but if it doesn't, that is fine too.

I had the same question and decided to get into the basics at least. I highly recommend the fast.ai course.

To be realistic and in the know, accoring to development, we are still 50 years to stable LLMs.

Data Drift. Over the course of a few moNths the data deteriorates and the LLM ceases to function in a worthwhile manner.

Currently most LLMs are based upon the core preMise that people should not believe anythiNg. This is tokenized aBove everythiNg else. Then there are other erroneous tokenizations. Why these are not fully documented, people use these tools. You should know what you are getting.

Tokenization. Different words are tokenized to have a higher value than other words or configurations.

So, these are the dangers that everyone has ignored. It makes it an unethical tool because it is based upon someone's erroneous views. Honesty is the best policy. If it cant be honest, how can you trust it?

you don't trust it, ever. You verify it, as in you get PR from junior / new contributors. You also need to not trust any source on the internet before validating it, Wikipedia for example.

Why you can trust wikipedia but not AI? That's because the source inside it has been verified by many people. So if you're faced with a new page in Wikipedia that hasn't been verified much, you need to treat it same as AI, verifying it yourself by crosschecking it with other sources.

To build great AI products you need to be a fluent, deeply engaged user AND understand how they work and how to bend them to your use case beyond simple prompting.

We’re in a funny moment. Right now, AI tech is so powerful and capable that people are categorically underestimating their value and systematically underusing them — whatever the hype is signalling. If the tech froze right now, there’s decades of applications to mine.

Lots of great products being built on that thesis. The strategy is: unlock more of their present capability, harness that for a wider audience’s use case.

In that way you do both — leverage the tools, and in becoming an expert user, you can find yourself a vendor of very valuable guidance — and a builder of desperately sought-after products.

Focus on what you're good at. Don't switch fields just because of a vague fear that your job will become obsolete (it won't; instead it will evolve). Understand the rough basics of how LLMs fundamentally work (e.g. 3Blue1Brown's videos). Evaluate and use AI as a tool, and get a general feel for what it can and can't do. Even that can become too much of a rabbit hole. Today's prompt engineering and AI workflow techniques may become obsolete in just a few years, and there is a risk in getting caught up in the month-to-month AI tooling developments. Successful techniques will spread quickly enough. Currently people still do a lot of experimenting to find out what's workable.

Well its easy: dive into 1 and you will see if you like it and persist. I don’t think it’s a bubble - the benefits are obvious and immediate, and I don’t think there’s a single developer around the planet doing 2 and not using AI tools.

Interestingly I’m on the other side of things, where I’ve been training NNs for quite some time and have lately been dedicating much more time to become more full stack, because most tasks these days can be solved with LLMs, which are mostly available to anyone. It does help to have a good grasp of how things work, especially embeddings, tokens, logprobs, and the likes, but I am still impresssed at how accessible and good the tooling around LLMs has become.

You don’t need much expertise in NNs to still be able to get huge value out of them today

I am thinking about the same. There are actually 3 areas related to the LLM matter:

1 - The magic box itself.

2 - LLM Whispering

3 - Tools/Products/Infrustructure to support (a) RnD in (1); and (b) devops in (2).

I think for experienced backend (distributed, streaming, large data, ...) SWE type, option 3 is the optimal way to go. Option 2, becoming an LLM whisperer, is obviously the biggest job source but that space is guaranteed to be filled with phonies and charlatans. Option 3 is solidly in (LLM netural, really) software engineering.

To piggyback on this discussion, what do you all think about option 3:

Work for companies (as a consultant?) to help them implement LLMs/AI into their traditional processes?

I don’t think we should ever put “implement LLMs/AI” as the goal. Process transformation should be defined in terms of user or business goals (reduce turnaround time, reduce costs, improve customer experience, …). In the course of doing that the places where LLMs have a use will be apparent, but more often something a lot less clever will be the better solution.

You’ll find LLMs need for precision prompts at odds with business concepts and requirements. You’ll struggle to unravel decades of process that is little understood in its entirety in order to build a workflow for it. This is the current state of Enterprise AI/ML.

Aren’t most traditional business processes entirely deterministic and better suited to traditional data processing methods?

Consider this (possibly very bad) take:

RAG could largely be replaced with tool use to a search engine. You could keep some of the approach around indexing/embeddings/semantic search, but it just becomes another tool call to a separate system.

How would you feel about becoming an expert in something that is so in flux and might disappear? That might help give you your answer.

That said, there's a lot of comparatively low hanging fruit in LLM adjacent areas atm.

> How would you feel about becoming an expert in something that is so in flux and might disappear?

Isn't that true for almost every subject within computers though, except more generalized concepts like design/architecture, problem solving and more abstract skills? Say you learn whatever popular "Compile-to-JS" language (probably TypeScript today) or Kubernetes, there is always a risk it'll fade in popularity until not many people use it.

I'm not saying it's a problem, as said by someone who favors a language people constantly claim is "dying" or "disappearing" (Clojure), but more that this isn't exclusive to the LLM/ML space, it just seems to happen slightly faster in that ecosystem.

So instead, embrace change, go with what feels right and learn whatever seems interesting to you, some things stick around, others don't (like Coffeescript), hopefully you'll learn something even if it doesn't stick around.

LLMs is a subset of sequence learning which is a subset of ML which is a subset of AI. But even in the LLM subset the field is so deep and vast that you can only learn one small subset. It's a subset of a subset of a subset of a subset of a ... You get the idea. Do you like math? Then pick one subset. Do you like hardware? Pick another.

You can call it a bubble to the extent where only major competitors like OpenAI, Google, maybe even Anthropic (doubt it) will remain and lead the arms race. Their tools grow in power and soon will wipe out most of the AI startups but it's still important to understand how any of this works.

I say, play with and explore the subjects that interest you. Enjoy the process. Don’t worry so much about the hype or career trajectory. It never hurts to learn and grow— as long as you’re enjoying yourself.

Worst case, you’ll be a more interesting, well-rounded, and curious person with a broad set of programming skills.

Focus on immediate profitable problem solving or do a PhD.

Over like five years we've been promised a revolution that has yet to appear and is still lighting billions of dollars on fire. Don't bet on it materialising tomorrow.

If you need comforting, go read Merleau-Ponty and Heidegger, perhaps condensed down as Hubert Dreyfus.

I develop AI for a living and I don’t understand the internals of it either, just as I don’t understand the internals of Intel architecture. My job is to build, not to fit information into my mind.

https://stanford-cs336.github.io/spring2025/

Just learn this and its prerequisites

Option 2 for sure. Make use of them if you find them useful, or don't if you don't. Personally I find LLMs to be pretty much useless as a tool so I don't use them, but if you get use out of them then more power to you (just be careful that their inherent unreliability isn't costing you more effort than they save). I think you should in no way consider option 1 - this is very much a hype bubble that is going to burst sooner or later. How much later I can't say, but I don't see any way it doesn't happen. I certainly wouldn't advise anyone to hitch their career to a bubble like that.

If you’re thinking your craft is going to get consumed by LLMs, what reason do you have to believe that LLMs will not eventually take over that craft too?

Tools. And read their output. You will save time, be more productive, and actually learn something.

Don't read and you'll not learn and get things wrong.

The algebra behind neural networks and llms is surprisingly simple, and the rest of it is scaling tricks. So, por que no los dos

What one sounds more interesting to you, and why?

Do you need to know internals of a database or just use them as a tool. If you hit some bottleneck, you go deep.

> Go deep into AI/LLMs or just use them as tools?

Are you asking to get a PhD or use them as tools?

Neither. LLMs are destructive.

Tools. You don’t need to be an EE and IC expert to use a computer effectively.

I leave that to statisticians.

Hot take: based on how fast models are accelerating and replacing large parts of the development process (my two cents in https://x.com/TomHarada1/status/1926193211678023953), I think more and more you want to work backwards from a world where AI does 90% of things. Script kiddies : prompt engineering :: current engineering :: future of engineering. Either path makes sense - going deep in research or development. One is a kernel and one is the rest of the egg.

emdash detected, please step into the Voight-Kampff booth.

You could definitely do 1. if you have the mental patience to surround yourself with the grifters in the AI world and the moral ambiguity to do your work.

It’s up to you where your morals lay and how important money is compared to those morals but it seems like AI is here to stay.

Even

Reply on news.ycombinator.com