There's no point at which turning your brain off will work

That _is_ valuable, and will remain so, but the temptation to turn it off is there because the loop is good and getting better at what it does. Once you're in the loop, with the rare high-value exception of catching total mistakes nad redirecting, you're mostly choosing between similar-yet-reasonable options. This isn't so much right-vs-wrong as relativistic optimization. Letting the loop do the work will get you a mediocre result quickly, and that's usually fine.

The most important time to use your brain is _before the prompt_. Once you engage with your LLM and agent, you start biasing yourself, and its reasonable suggestions constrain your visibility into other options, other worlds.

First, use your brain. Then write the prompt.

Even if I agree that you have to use your brain, when my boss is literally telling me to stop questioning whether AI automation is the right answer in team meetings/chats I'm just going to shut up and give them what they want.

Then again, for many people their job security doesn't depend on what work they do, it depends on how well they are embedded in the organisation. Before AI you also already had plenty of people doing their job at a questionable level, yet never seem to get fired.

New concept: the AI patsy

If you are using these systems you still need to know what you want and how to specify it. You can use agents to automate much of the work but it can’t tell you that your ideas are bad, that there’s a better way to accomplish your goals; it can’t tell you what you want. And if you can’t understand what it is outputting you’re failing in your primary task as a developer: understanding the problem.

I agree with the title. I just don’t think these systems will ever be capable of replacing humans. As much as product managers and executives would love it and not have to deal with us; they’ll just be shooting themselves in the foot if they try to replace professional developers with AI harnesses and fire all the people with the institutional knowledge and skill.

Company is pleased! Cost savings!

Alas, the joy doesn't last long as pretty soon after the company realizes that after an LLM replaces almost all their employees, the much easier task of replacing the entire company begins, because the company itself has just become a much thinner meat proxy for getting an LLM to do whatever it is the customer relies on the company for.

Customer is pleased! Cost savings!

Alas, the joy doesn't last long as whatever savings they have from replacing the company with their own LLM loop is offset by whatever it is they do to earn income being similarly impacted.

All of the moats degrade together though they may fully give out at slightly different speeds.

     Let's say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There's no point at which this methodology will work for the employee

The human's purpose is to keep an eye on the LLM, occasionally intervene/interfere with its work, and most importantly, incur social/legal liability for anything that the LLM does wrong.

Kind of like the backup drivers used by self-driving cars.

Calling someone a "meat proxy" for being the human in the loop is wild.

The problem is, that digital work, the new frontier of employment for a lot of epeople, got and will continue to get A LOT cheaper, faster and more accessable.

The PO who always complains about someone? They will just vibe stuff. It might even be shit but it will work good enough until LLMs/AI/AGI/ASI is good enough or better anyway than an avg developer.

Man i have seen so so much shitty code even in software companies.

You know when your HR software takes seconds to load? Yeah an LLM can do that today too

What's difficult, is I now spend most of my time on these high-throughput tasks, where my brain isn't exactly off, but focused on quantity over quality. It's often difficult to identify when a task requires "real" brain-power, and when it does, it adds a lot of friction to dig into the code and orient myself. So it's hard to know when it's the right time to do that.

If this is true, turning off the brain... is completely rational. Why burn out the brain for something happening either way? It's not like being fully engaged is going to result in anything that much better for the employee. In a world where code is this disposable, memorizing a codebase that won't recognizably exist in 3 years is also stupid.

Here's the thing I'm finding - LLMs allow me to do more coding when my brain isn't firing on all six cylinders. It means I get more code written because I don't have to be "in the zone" to exhaustively hunt down every piece of API and dependency and have my flow continuously broken up by missing information. The code written in that state is lower quality, but I can patch up five-cylinder-brain code faster than I can synthesize code from a blank page outside of my flow state.

My solution is saying “gnite” and coming back in a new terminal window with a clear restatement of my purpose. Or “what are you working on and why” can help.

Terminal windows are not great for keeping context; waiting for better design. It is possible.

I was dreaming to be promoted to a cushy management role where I could talk bs for a living.

I managed to attain 'hands-on team lead' and later an ill-fated 'bootstrapped cryptocurrency CTO' role... Nobody wanted to pay me to talk as a career. Now they do!

So I'm very happy about that.

I feel contempt for all the people who had it easy and 'chose' to be engineers voluntarily.

Even with small things, you need to be at the wheel. The other day I had to stop copilot from literally deleting the file I was working on. Why did it choose to go down that route? I don't know, but I nixed and reported the chat once it did that.

LLMs are dangerous and if for some reason you are stuck with using them, do NOT ever turn your brain off.

Ideally, you should not be using LLMs at all.

But when the human suggests changing the course, the LLMs will often agree the new direction is better and start heading there...

If you can do your job while watching TV your brain is turned off.

Another problem is that when something breaks then I am very limited in my ability to reason through the breakage. It's necessary to go back to the LLM and ask it why it broke. At which point, I'm able to follow the reasoning — but not in any durable manner where I'll be able to remember it. To me, it's sort of like the difference between memorizing a passage and having a ready reference at hand at all time. We had this same problem with information before — now it's come for our reasoning.

Of course, this becomes a lot more difficult when you're unemployed.

Then there's the software that I write for a living, and almost my entire day is now spent making hard decisions. /grilling gives me difficult technical design decisions endlessly and I am forever doing product management on features from agents. My brain is back in CTO mode, which isn't what I'd planned for myself after I'd left that particular cursed path, but fine. I am doing the opposite of turning my brain off, because I no longer can just chill out and mindlessly write code.

Both these states can exist simultaneously. But my brain work has shifted into something different, while getting no less intense.

Why do you think management is so gung-ho about LLMs? Employees are expensive, and laying them off is therefore extremely appealing.

The entire, explicitly stated goal of the AI movement is to produce an ASI, which is better across every metric than a human. Even if you refuse to turn off your brain, the economics of paying you to keep the lights on upstairs aren't going to work out.

This was never going to work out well for the human.

I'm not sure what else to say. You listened to these people speak, what the heck did you expect?

he was writing in reaction to the McCarthy era which, arguably, is very similar to kind of bombastic nativism we're seeing now from our political elites. his argument essentially being that there are cultural and institutionalized forces that are driving people to, well, turn their brains off - specifically to no longer be skeptical of what they consume, to not want to refine and improve their toolbox of analysis, to not want to understand how things work for the sake of knowing

and that these acts were deliberate - a populace is so much easier to control when it's complacent and unknowing, who believes, unskeptically, that Oceania has always been at war with Eastasia, unquestioning about who was writing the PR missives that feed the news cycle (see also: https://www.commondreams.org/opinion/what-is-copaganda), how verbatim they are, and what the real intent of the authors were

like, I get that you don't want to put in effort for a job you don't like. but that effort also makes you better at your work. it makes it easier on you, it lets you create less incidents, it empowers you to be more agent in your own life, to, who knows, have the guts to start a consulting gig because your skill-sets actually are better than the average

there's so many benefits from just being more diligent, longterm, proactive benefits. but that's not everybody's extrinsic motivator and it's so much easier to just be Skinner-boxed by social media companies into doomscrolling, I suppose

Developers like to "turn their brain off" during "super hard debuging" sessions. Mental flow state is a well documented as pleasurable experience. It is like a drug!

Auditing and reviewing autogenerated commits, is hard gruelling mental labour, compared to that. You do not even get a credit as an author! You can not get into groove to get your fix!

So "there's no point at which this methodology works for the employee" reads as true to me, but the thing being punished is the brain-off part, not the using-an-LLM part. The person supervising agents and cutting the diff to 25% (the Gary Bernhardt example) is doing more valuable work than they were pre-LLM, not less.

”Nowhere is this clearer than in entry-level software engineering—though the picture depends on whom you ask. Brynjolfsson discovered that employment among developers aged 22 to 25 has fallen nearly 20% from its late-2022 peak. The online job site Indeed also paints a stark picture: software development job postings have fallen 53% from the same starting point.”

https://insights.som.yale.edu/insights/the-real-job-destruct...

They aren’t firing the meat proxies to that degree yet, but they absolutely have stopped hiring the meat proxies. It’s going to be a very dirty year on Wallstreet when you’ll see massive sneaky layoffs of the existing meat proxies.

You can turn your brain off with these models. It is what it is.