Hacker News Reader

Comment by dofm | Hacker News Reader

The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it.

I don't know how much serious hands-free agentic coding I will ever do on my MacBook alone, but I do know that I would not have got so far into understanding this without tinkering with local models, llama.cpp, LM Studio, and LM Studio and all that.

I totally struggled to find the right frame of mind to explore any of this stuff without feeling defeated and bamboozled. Because it's just huge, exhausting, jargon-drenched, unknowable, and I am over the hill at fifty-plus.

Until, that is, I could poke around with setting it up on my own (secondhand) machine, watching the API calls, understanding some of the terminology. I didn't even buy the machine for that; it's just adequate to the task.

The Neo is too small to really get much benefit from this opportunity to make it more visceral and knowable.

View on HN

> Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it.

Cloud models are (much) faster, they don't consume so much power/generate heat, they have much bigger (LLM) context, they're much more precise and they have a much wider (engineering) context of the given problem.

Except privacy and use cases that are blocked by cloud models (e.g. reverse engineering), local LLMs are currently an expensive toy.

When I try to program with a local LLM (I'm on a 32/128 GB system), I end up wasting time compared to a cloud LLM.

Again, I would not argue against any of this.

And I can't say that I won't switch to openrouter (even just for the same models) at some point.

But one of the things I have found about my own process learning is that some lessons only come to you when you make yourself available to them. And if that means doing things the difficult way, that is what you should do.

Difficult... and wastefully expensive

I mean, it's a (secondhand) computer I bought for other tasks (processing very large photos, compiling large apps quickly). It's running all the time. It can also run LLMs when I want to.

The rest of my life is ultra-frugal so I am relaxed about this.

Don't bite. You're right.

Having spent a good weekend learning how to perform latent-steering through playing with pytorch and a local Gemma4 model, there is no way I could have groked any of that in the the way I did without hands on time.

This is on an M3 Max 36GB I've had for a couple of years. No further outlay needed.

My thinking is totally aligned with yours, perhaps its because I am trying to do a second act at almost 50 from blue-collar to white collar office work. I have no formal degree, but I have been hobby programming for 20 years. I have made a habit of "letting myself be available to all lessons"... the localllama group has made this journey really fun if nothing else. I have learned an ABSOLUTE ton from this era!

I have been contemplating a move in the opposite direction because I have just been exhausted and depressed, so for me, really learning this stuff this way has been about managing those feelings, about a sense of pride and ownership of my processes.

I don't know if it has changed my mind about a career change but as I am sure you can understand, I no longer feel like I am running away defeated.

My very best wishes to you :-)

Seems like an investment into building expertise, which is likely to have high ROI in the future, rather than a wasteful cost.

> Seems like an investment into building expertise

Learning LLM internals != using them as daily tools (the latter was the topic of discussion).

Shifting to the learning topic: no need for expensive hardware. Even a 16 GB GPU will do, and that's (relatively) cheap and available.

People pay thousands for model trains, everyone needs a hobby.

Training models vs modelling trains

Ah yes, the EMD0E9-30B-Union-Pacific.gguf

I'm sending Codex-gpt-5.5-cyber.stl to my printer right now!

The key word there is 'currently'.

Economies of scale are a fact of nature and aren’t going to be subverted in the future by even the most advanced local models

Which is of course why, if you want to render 3d scenes to play a video game, you have to rent time on a mainframe system. I don’t see that changing ever - it’s just economies of scale!

(sarcasm, btw)

Things can get both more expensive and cheaper at scale, hence the term.

For example (and relevant to AI) I can generate electricity on my roof at $0.20-25/kWh, batteries included. In California the electric utility can’t offer it cheaper than $0.30-0.50/kWh. Therefore at scale, electricity is actually more expensive.

There are many such examples.

Yeah, I think the fallacy here is the conflation of scale and centralization.

Right now, there is way more scale in centralized AI than there is at the edge. But that could flip. I'd still probably put the probability that it will under 50%. But I'd also put it above zero!

Apples and Oranges. The utility uses a weird conflated fee that combines the price of the electricity and the price of connecting your house to the grid. If they split it up your marginal price per kWh would be much less.

... said the IBM executive to a young Bill Gates.

The economies of scale gains are lost because you still have a middle man hosting provider who wants to profit too.

Over the long term it's always been better to buy than to rent, even if the renting option is technically more efficient on the GPUs, you don't have to pay some hosting providers profit margin.

If the hosting provider can fit 1000 users onto 100 GPUs, that's enough for quite nice margins and being far cheaper than buying your own GPU.

And for users that aren't running multiple agents 24/7, you should be able to fit a good user:GPU ratio.

Maybe. The economics work out better than for game streaming. When I looked in to game streaming it ended up being cheaper to buy over the long term. Though games tend to use 100% of the hardware for hours, and they tend to all be used at the same hours of the day and have to be hyper local for latency reasons. Something LLMs don’t have issues with.

Setting aside that very little about economics rises to the level of "facts of nature" like physics...

What makes you so certain that economies of scale won't work the opposite way you imagine? E.g., if model improvement tapers off, but RAM costs decline (hard to believe atm, but historically likely), then eventually everyone will be able to run SOTA models on their personal hardware.

Heck, even if model sizes simply grow more slowly than RAM costs decrease, the same would happen.

Anything done local will likely come at higher cost and at scale with less energy efficiency and commodity, with less possibility to fine tune engineer deeply on wider horizon of issues.

That's never the point of keeping local alternatives though.

Right.

For me this dates all the way back to installing Slackware 1.0 (0.99pl12!) on an offline 486SX rather than just using the internet-connected workstations in the lab.

Here, I already had a Mac that was powerful enough to run a local LLM, so now I do, because I can.

> Cloud models […] don't consume so much power/generate heat

I do realize the cloud is just someone else’s computer right? Power goes in, tokens and heat come out - just in another place

The cloud computers produce more tokens per watt. That said, if you have a computer at home running 24/7 for other reasons and you also can use it for some LLM work, why not.

> currently

The interesting question is whether that gap will narrow, and if so, how much, and on what timescale.

The exact answer to this question is not knowable, but if you are the kind of person who comes to a site called "hacker news", and you think there is a nonzero chance that the answer is that yes, the gap will narrow and this won't always be an expensive toy, then now seems like a pretty great time to get in the game and start exploring the capabilities.

The same applies to ray tracing; the question whether shadows can be rendered accurately and cheaply is also not knowable - but it's very unlikely that ray tracing will become as cheap as rasterization for the same quality target. LLMs are pretty much the same; they're inherently computationally expensive, even if optimizations are in continuous development.

I agree completely. I think local AI is best limited to purpose built SLMs; all this craze around running quantized coding LLMs has taken the attention off SLMs.

Same. Local LLMs are fun to experiment with, but when I want generated code of a sufficient quality, I use a cloud LLM.

From your post I can only perceive the instinct to pick a side, and trying to make sure it is the "winning side". But the truth is far more nuanced. I have acces to both, paid and local models, and even if slower, the local models have been far more educative about how these technologies are put together, and what is required for local computing to thrive again. Paid models will not suddenly disappear just because I play with glm-4.6 on Ollama. At the same time, my work pays the cloud subscription and I use the cloud models to perform the tasks my work requires. There's no need to choose one side.

the downsides are also: It's underlying "alignment" can change at any time, it's non-determinism is manifold, because of the alignment and the entire aparatus behind it (currently claude's KV cache use is turning it into a $$$$ burning machine), it has zero customer service expectation, and they can ban you at any time.

So if you're just putzing around, sure, cloud will be fine, but if you're building a workflow/business around these things, you're essentially making yourself an indentured servant.

Exactly. The distinction between the various layers in "AI" systems is pretty vague to the newcomer. What is the "model" vs. the engine "running" it vs. weights?

I don't recall any previous tech stack that was barfed onto the scene with so little background or reference material, going from zero to endless undefined jargon... and no primer in sight.

For people who demand an understanding of their tools, it's a lot of work. I recognize the value of "AI" in performing the tasks I'd have to do manually; for example, keeping the data structures of my front- and back-ends in sync in a project. But do I want to interrupt my development and take weeks off to digest all of these tools?

And if I do, I want to run the show and fully understand it. And like you, I think that's best done locally.

For the most part you can just download LM Studio and go from there. It provides a chat interface and an easy-to-use interface to browse, load and use LLM models. The engine: it is abstracted away by LM Studio, if you want to dig deep it's llama.cpp as the runtime. Weights are the files what you download, they are the models for practical purposes.

I definitely would recommend LM Studio as a learning environment, because it surfaces a bunch of things in relatively clear-minded ways. I am very grateful for it.

The most unexpected thing for me was kind of philosophical in a ‘holy shit’ way.

Cloud models still feel ‘magic’, like you send a request off and get something back, like it’s something ‘special’. I used to joke that ChatGPT might be some kind of mechanical turk underneath.

Watching a model run local on your own machine hits different — you realise that yes, it IS just a computer program. Which for me actually makes me appreciate the leap we’ve made MORE, not less. From an information-theoretic point of view, LLMs really are something special.

The fact that they are just programs, that I’ve now experienced first-hand that they’re just programs, makes all those questions around consciousness and intelligence much more interesting.

Yeah, it's been fun for me running models (mostly Qwen 3.6 27B) on my 48GB M4 MacBook Pro. When i'm using it to run models, it's basically unusable for anything else - I actually do the work on my Macbook Neo. Took me a while to figure out why the models couldn't figure out how to make tool calls - because LMStudio by default uses a 32K input window, which is smaller than OpenCode's prompt, so half of the instructions were being pruned from the middle!

Yes — there is a setting for that isn't there. And as soon as you realise there's a setting for that, you have new knowledge.

Qwen barely needs any of Opencode's prompt, in my experience; I think I cut it down to about three general lines I found by googling. Mainly you need only a pre-amble to make sure that the plan mode, plan switch and build mode prompt fragments make sense.

Gemma 4 also needs almost nothing at all, which is fascinating, considering it is not a coding-specialist model. It just seems to be who you need it to be when you ask.

What are those 3 lines you've cut it down to?

I've been using pi-coder the couple days with Qwen, and its minimal prompt works MUCH better than OpenCode's bloated prompt.

I was going to reply with my answer but I tested it again and realised that actually in some cases the custom prompt is somehow being ignored, so I don’t truly know how good it actually is in opencode.

Poking around in the convoluted opencode source or the overstuffed github issues to try to make sense of exactly what is happening has convinced me to switch to pi as well, simply to have a thing I can more fully understand. I am using paseo anyway for the GUI on the Mac, so switching agents is easy enough and pi has always seemed like it might be the right choice in principle. Initially I just picked opencode as the easiest path but now it feels like the time.

That was basically the conclusion I came to after I spent a couple hours trying to puzzle my way through opencode's settings - extremely opaque and hard to tell what's happening. Pi is simpler, and i figure if I really need a better system prompt, that's literally what AGENTS.md is for, right?

Yep — it hasn't changed how I feel about what LLMs are capable of (and very much not capable of) but this visceral feeling is fascinating.

Like, just watching a computer I already owned act like ChatGPT with the wifi disconnected.

It was the first time I stopped feeling quite so helpless, somehow.

> I totally struggled to find the right frame of mind to explore any of this stuff without feeling defeated and bamboozled.

I found LM studio to be a nice starting point. Frindlier and more featureful than Ollama and not as intimidating as llama.cpp (though you will want to use that eventually)

LM Studio is also nice because of the way the interface explains things; parameters have explanations and hints. It has been designed by people who really care about making it understandable.

I tried Ollama but I've settled on Unsloth Studio generally; once things really settle down I'll just run the llama-server UI, which is pretty nice.

A friend is tinkering with LLMs for amusement on a 16GB Raspberry Pi 5, and when I explained that llama.cpp now had a typical web chat interface he was so happy — it's amazing what the "table stakes" are now.

I agree with the learning aspect, but I have another motivation. I suspect that closed models might become too expensive to run for personal hobbyist use. I’ve been planning to buy a 64GB machine just to allow the limited local models this enables.

I've setup to local paradigms for local coding:

- opencode with it's webui

- deer-flow with it's research/powered front end

They both run websites so you don't have to baby sit them (eg, keep your mac open). I've build a pdf compressor over a few days by first having deer flow try and research the frameworks and pipeline. It stalls out because its not really a fluid programmer. Once it stalls out, I transferred it (manually for now) to opencode and it's refactoring it because it's just a collective bundle of sticks and it needs a lot of testing to tweak out the limited scop context. LLMs can't really hold large scopes (locally anyway, from what I've read from HN, it's possible with longer context).

It'll complete in a few days with maybe 3-4 hours of full attention interaction, but it's running 3x that without my attention. Obviously, if I paid more attention it'd run quicker, but since it's local, it's not pumping out large volumes of code, it's mostly looping over tests and capabilities as observed.

It's running Qwen3.6 35B MoE on a AMD 128GB strix halo. If I switched to the dense models, perhaps it'd be smarter, but the trade off seems to be much slower gen.

> - opencode with it's webui

Have you tried Paseo?

I have opencode in a VM, and the paseo daemon running in the VM, and then the Paseo Mac app. Really nice.

(You can also use the Opencode GUI to frame a remote opencode web interface)

You can also just add OpenCode web as a PWA, if that's what you mean by "frame".

I'm gonna check out paseo, but am not looking forward to all the ram the agent needs + all the ram paseo needs

Have checked out Paseo, not sure what it offers over opencode web though. Definitely seems great if you're using other harnesses, but it seems like all it has over opencode web is split views and native apps. Neither of those really matter to me, plus you lose some opencode goodies. The preview urls are a neat idea, but our dev servers at work are mostly port independent and required to be on a certain subdomain for auth.

Thanks for posting this. This is the tinkerer mentality. It is not for everyone, but certain things can only be learned in that way. It is the best antidote to AI paranoia. There is much that does not transfer between frontier models and local ones. There is that. But you can not tinker as much as you can with the former.

> The maths there is pretty undeniable, but it is not where I'd make the split. Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it.

Seems like a GPU with 12GB+ VRAM is going to be a much more affordable way to achieve that? Even a B580 should get reasonable perf there.

No idea. I am a Mac guy, have been for a very long time. I buy them secondhand as a rule.

I guess I would build a powerful home LLM server if I was convinced I really needed one for my purposes for some agentic application or other. At the moment I'd prefer to ride this out with a machine that is also an excellent Mac.

It's also great to have capability to run local models for more brute force tasks. Because you can change the system prompt, you can get local LLMs to do all kinds of high volume tasks without burning through tokens on a hosted model.

Just one example, I needed a bunch of images tagged and organised, with a local vision capable model I could pretty easily set that up and leave it running overnight.

I already had the GPU and memory for gaming, so it was at no cost for me to start running local models. But I feel the long term writing is on the wall, local models will only make more and more sense as they get better and more efficient.

> Having a machine that can run some modest local LLMs, like the Gemma 4 12B, is really worth it.

Agree having a powerful machine is really worth it in general for professionals, but strong disagree that running local LLMs has anything to do with it. It's hard enough as it is getting a good ROI on your time/money prompting/wrangling with frontier models. IMO leaning on the comparatively limited capabilities of local LLMs is best avoided in favor of keeping your own personal coding skills fresh and continuing to learn new ones.

I'm not that bothered about my coding skills, which are fine, and pretty up-to-date considering I'm now an old bloke. I am bothered about building an instinctive understanding that helps me deal with my anxieties and decide whether I want to carry on with this working life or quit.

I needed to do this, this way, in my own time, to put my brain back together. It has worked for me, which is why I recommend it.

YMMV.

Unfortunately the local llm bunch is not the most emphatetic one in my experience: you are somehow "expected" to immediately know all this stuff and god forbid you ask the wrong question. I've never seen or felt this level of bullying and weird vibes over tools and LLM models. "My setup works for you or beat it".

There's also a lot of cargo-cult stuff, isn't there? Especially in the Reddit groups. Just do XYZ. And people ask why and they are never around to explain. Because, perhaps, they can't.

(Very reminiscent of 3D printing, where you get a lot of very trivial advice poorly applied, which is an analogy I've now made several times.)

Several of the youtubers are pretty helpful, though; I watched half a dozen things and absorbed the broad pattern and then went for it.

Also I got a lot out of reading HN comments, which is why I am here; tucked away in the corners of these discussions are people who can help. Over time I hope I am one.

Where has that been your experience? My experience interacting with people about this is almost entirely in HN threads like this one, and I haven't found what you're saying here to be the case.

But if this is the case, as you say, it seems like a good opportunity to build a more welcoming set of entry points into this!

Continuing to learn new ones, like what?

To me, "how do contemporary AI systems work and interact with contemporary hardware and how can I best take advantage of their capabilities?" is the set of skills that are worth learning at this moment.

What else is there? New / additional programming languages? New / additional database systems? frameworks? orchestrators? cloud provider / infra tooling? architectural patterns?

I dunno, all of this seems really boring and "been there done that" to me at this moment in time!

Yes, that all tracks, and all of those skills are worth maintaining and improving. Great to tinker with LLMs locally hands-on to learn, and having a powerful enough machine to enable that to a reasonable degree is just one of many reasons why it's worth it. I'm just saying that IMO "how can I best take advantage" lands firmly in the bucket of only cloud-hosted frontier models being worth my time. I would speculate that holds true for a large portion of the wider HN audience but YMMV of course.

Maybe. I felt this way a year ago and definitely two years ago. But now my sense is that it's played out at this point, and the valuable thing to build expertise on now - precisely because I think it's coming rather than here - is local / open weights / hybrid models and harnesses.

I just got Claude to download and install all the models and servers and agents and prepare all the launch scripts for me... no need to learn, just ask it to do it for you

Right, but I am a middle-aged bloke who is experiencing existential angst about whether I can carry on in this industry.

I have a pretty deep, maybe paranoid need to be confident I have an intrinsic understanding, and I have found in my life that lessons come to you when you make yourself open to learning.

So I need to build on top of what I know, taking as much of the hard way as I can bear to take at any one time — it has to be not quite difficult enough to put me off.

I can't really explain what I have learned this way that is different, but I feel it in a way that I wouldn't if I'd simply pushed a button.

For the same reason, I have a really basic 3D printer that I've set up myself, set up Klipper, configured how I want it, learned how to calibrate, all that. And now I can say that I feel I have an understanding of 3D printing. I could hold my head above water in a discussion with a real expert, maybe find work in an adjacent field where my insights would keep me grounded.

I can afford a really good printer that has all that set up, and more, has no problems. But I'd just be someone who has a 3D printer.

(Also who am I kidding about the existence of a printer with no problems)

This really resonates with me, and I'm only a decade and change into my career. I use claude a lot day to day. I try to use it sensibly, making me more productive and produce better work. I'm also trying not to lose understanding along the way. I want to be able to actually talk to the conclusions I'm reaching.

I have colleagues that seem perfectly content to delegate too much to the agents, and it saddens me. It feels like there will be swaths of engineers that didn't train some of the critical thinking skills that I take for granted.

I certainly see it in slack discourse around anything more complicated than a feature implementation. Maybe I'm just cynical. Time will tell, I suppose.

For me (about halfway between you and dofm in my career by your own statements in this thread), it's a dream at the moment. I can delegate all the tedious stuff that I've done "the hard way" a thousand times already and feel I have very little of value remaining to learn, so that I can spend more time on all the things that are actually new and thus much more interesting.

It's been a great multiplier for me in similar ways. The "dreamiest" thing has been that it has freed up time that I would normally have spent doing sprint work, to work on things that just don't make the cut until it's bad enough to deprioritize other work.

Over the last few months, I've been digging into performance problems with a high throughput service that my team owns. I started working on the problems in my own time, put out short and medium term improvements that legitimately avoided operational issues, and started developing an alternate architecture that should meaningfully address the problems for the long term.

I've learned new things and made improvements that probably wouldn't have ever gone in otherwise.

Yes exactly. There is a narrative that it's driving everything toward low quality slop, but in my own work it's exactly the opposite. We're doing work on quality and performance that we never would have gotten to in the past.

I've spent my whole career being frustrated by the pile of low severity bugs and performance issues that "I could fix that if I could only justify putting a couple hours into it!". And now I can just fix all those. Nobody is going to question my use of time to write prompts and do code reviews of those things, when I can to my "real" work simultaneously.

You will not live enough to learn everything. Eventually you have to say "I could figure [something] out but I won't take that time." Most things are that way - I probably could learn brain surgery (I used this example because it has a reputation of being a very difficult course of study). I would like to make a lathe from scratch - but I don't have easy access to enough iron ore to get started - even if I start from scrap metal, I probably wouldn't spend months making my own surface plate (...) and so I own a factory made lathe instead.

That is why I'm content to delegate to agents - I have more code/features I want to write than I have time to debug (writing is the easy part).

Yeah, this is just the engineer's mindset. It's not surprising that this is a popular view here, even if it is not (and does not need to be) the mainstream perspective.

> mainstream

What does "mainstream" refer to when we're talking about software development and LLMs? As opposed to "engineers".

This is a very fair question! When I wrote this comment, I was definitely thinking of the "real" mainstream, i.e. users of llm chat to generate text, not software engineers.

But I think there is (and has always been) also a distinction between the "mainstream" of software developers vs people who are working on new tools and capabilities to be used by that "mainstream".

IMO it is certainly true that the most efficient and cost effective was to do "mainstream" software delivery at the moment is hosted frontier models. But for people thinking about "what's next?", it makes a ton of sense to be exploring different models in anticipation of a possible (but certainly not inevitable) sea change.

Then what is the point of ddalex?

I think if you really don't feel the need to know the "why" of everything, sometimes this might be the right approach. It is quick, pragmatic, gets you started.

Maybe my biggest problem with the world of agentic AI, and the reason I am putting myself through learning it the way I am, is that the need to know the "why" of everything is so fundamental to me, that I don't know if there is any point to me without it.

So this is really the only way I know how to proceed.

To me, this is just a question of specialization. Not everyone needs to be a "I understand how the system actually works" person. In fact, not many people need to be that person. But every system does need some of that person to exist!

And we happen to be discussing this on a forum where the type of people who will be the specialists for the kinda of systems we're discussing are likely to gather.

I'd be surprised if in my casual discussions out in the real world, I were to run into a lot of people who care exactly how all this works, to the extent that they want to invest significant money into hardware that allows them to run things themselves and dig into what's actually going on. But I'm not at all surprised to come across such people here! (Indeed, it would be very disappointed if I didn't!)

I think the more you know of how (many) things work, the slightly better you'll be at using them. From dishwashers to CPUs, from car engines to watercolours, from guitars to kitchen knives... You get the gist. Once you internalize a model of the thing, it becomes closer to an extension of you than a tool. You drive it better and with less friction.

Yes agreed, but there is limited time in a life, so there is a fairly high opportunity cost to internalizing a model of many things, which scales quickly with the complexity of those things, so people rationally limit the number of things they invest their time in. For the vast majority of people, I think it makes a lot of sense for AI systems to fail to make this cut. But for most of us here, on a site for computer technologists, it almost certainly makes sense for us to learn as many of the details as we can manage.

>no need to learn, just ask it to do it for you

And that's how skills die.

When's the last time you shoed a horse?

The reason I delegate so much of local LLM installation and administration to Claude Code is simply because there's no point learning practical things that will work completely differently in a couple of years, or in memorizing procedures that I'll forget long before I need to perform them again.

No longer having to sweat all the details is a Good Thing, not a Bad Thing.

I am not sure I disagree, and I certainly don't mean to disagree very fervently.

But I think if you want to really learn to ride well, understand horses well, there might be some benefit in learning how to shoe a horse. At some level it should never only be someone else's job.

At the same time, most people can drive without understanding how a car works.

You actually do need some understanding of how a car works, no?

For example, you need to know it uses gasoline (or diesel), it requires oil changes every certain amount of time, break pad replacement, etc.

You also probably need to know that you can't operate cars over a certain amount of water, that you need a driver's license, stopping at red lights, etc.

Sure, you might not need to be a mechanic, but that's far from not understanding how a car works, which to me sounds similar to knowing how to shoe a horse, which is different than being a horse vet.

Yes, and they're all the worse, more at the mercy of car companies and mechanics, and less aware of the world they live and operate in, for it...

Having to shoe a horse never was a general skill.

Maybe a more apt analogy would be a skill like making fire without a lighter.

Writing software never was never a general skill either though? Or am I misunderstanding your point?

Yes, LLM are thrown through pretty much everyone digital life whether they like it or not, it's not just devs. It might even unlock exploring things that need code that average user wouldn't have dared to do before.

If I worked with horses for 8 hours a day I imagine the answer would be "recently"

>When's the last time you shoed a horse?

That skill died too, so what's your point?

Skills sometimes do that. What's your point?

Skills are good. They shouldn't do that.

It did not. There are professional horse shoers like there always were. Not in huge numbers, but there are. Domesticated horses didn't just disappear from the world.

Except with AI models it's possible to make a backup of them creating a permanent artifact of a skill.

And why is this skill important, if a machine can do it ? What's the last time you ploughed your field with oxen ?

The skill isn't the plowing. The skill is thinking and learning and the ability that atrophies is that of mental effort (which is what drives thinking and learning). Losing those will affect people's lives and potentially even their humanity.

I don't necessarily think your answer is wrong for all people, but if you work in software... how do you plan to differentiate yourself from everyone else out there, if the depth of your understanding is "Claude can do it for me"?

This ultimately is the discussion I am here for.

I mean one of the things I use a local LLM for, because I can, is to generate starter documentation. But I ask it to — I want it to give me overviews, plans, all that. It can make something bespoke for me.

I guess I could also ask it to do the work. But where do you draw the line?

The universal labour-saving device is the great provocation of the next 100 years I think, and both Star Trek and Wall-E have grappled with it.

I'd say give it some time for the dust to settle. This field badly needs standardized benchmarks even before the conversation around model goodness can start.

> I totally struggled to find the right frame of mind to explore any of this stuff without feeling defeated and bamboozled. Because it's just huge, exhausting, jargon-drenched, unknowable, and I am over the hill at fifty-plus.

Hello, my brother, just know that you have a fellow passenger in life at the same age who thinks the same thing. I agree that the local stuff is helping my understanding a LOT.

However, my gut feel as someone who got to experience the TeleBomb after the DotBomb is that the obfuscation is INTENTIONAL--it's neither you nor your age. I remember asking people to explain to me what the OC-768 startup endgame was when roughly 10 OC-768 links could carry the world's traffic at the time--and everybody giving me blank looks. The AI Bubble has the EXACT same feel as the Telecom Bubble--just bigger.

What I really wish is that I could find a VPS-type provider where I could toss things into their NVIDIA/AMD machines for an hour or two. Alas, all of the providers seem to want massive paperwork and huge minimum purchases.

I can't wait for the bubble to pop so that we mere mortals can finally build with this stuff.

You looked into vendors like https://www.runpod.io/?

What’s the use of a 4gb Gemma other than to just play with it ?

Honestly your best bet is to buy a $20 Claude subscription, ask Claude to set it all up with Pi and llama.cpp and come back in 20 minutes after a cup of coffee. This is also a good idea because it will help set expectations of what a local model can do vs. a frontier model.

This is what I did after struggling to get llama.cpp working at a decent speed on my M1 Macbook. The secret is to very specific with your needs and targeted in what you are using llama.cpp for. Mine setup is just about strictly for qwen3-coder and now, I get a fairly decent speed out of it. I also installed Cursor to check Claude and it all worked out well.

I don't know why we're even talking about Qwen3.6 for writing code when qwen3-coder exists. My experience is there's no contest. I'm using 30b with 96k context on a dedicated server.

For agentic workflows like tool use, editing codebases, multi-turn debugging?

Are you talking about Qwen3 Coder 30b a3b Instruct from August 2025, which is a non-reasoning model? Or the more recent "Qwen3 Coder Next" from Feb this year with 80b params, 3b active? I found Qwen3 coder next to be quite good on openrouter [1], but couldn't run it locally.

[1] https://openrouter.ai/qwen/qwen3-coder-next

Reply on news.ycombinator.com