One Month Without AI

Over the past six months I tried using Claude, chatgpt, Grok and Gemini. At best I got reminders of how things worked. People online say they use them to write their code. The code they supplied to me has NEVER worked or was so convoluted that I threw it away and did it myself.

At most, I use these tools as search engines. Even then some references are poor.

I'm starting to think this is becoming a sad, sad world and AI is just the new TV of the programming world.

Just today it made three glaring mistakes in one session:

1. It read a file in the wrong directory, because that file had the same name as the file in the right directory. It apologized when I challenged it, promising me that it would remember to "read import statements" in the future.

2. It miscounted the number of times a function was called in my repo. It said 20, while my built-in IDE search accurately showed 17. Again, it apologized when I corrected it.

3. It referred to a variable by name that does not exist anywhere in my code. It apologized, and said it was referring to a variable used internally by one of the third-party packages installed in my repo.

So many apologies.

It's the little things like this that remind me on a regular basis just how little I can trust artificial "intelligence."

The real challenge is forming the new ideas in the first place and most of those new ideas coming either from using the code as a product or time spent maintaining and refactoring large code.

Anyways, if you want to continue on the path towards regaining control and take it to the next level I wrote something similar here: https://blog.sharefile.systems/be-brave-go-low/

This is using AI for productivity in any domain, in a nutshell. I just wrote a book using Claude as an experiment, and while the thing got done and it was an amazing tool and a great experience, what I’m left with is a book where every line needs rewriting, there are logical inconsistencies throughout, and the style is so bad it should actually just be binned rather than rewritten.

Playing an instrument vs electronic/computer music.

We do forget skills we don't practice, especially fine motor skills (like playing the guitar or typing code).

There's inherent pleasure in playing a musical instrument - practicing improves fine motor skills and produces satisfaction.

You can play for yourself and that can be a great experience.

Often people create music for other listeners - and now the satisfaction comes not just from your skill, but from how the music impacts your listeners.

They say you can put more of your 'soul' into music made with an instrument, but I'd say there's quite a bit of electronic music with just as much soul.

People who create electronic music don't generate any of those sounds with their fine motor skills, but they do have a plan about how the song progresses and what emotional state it elicits in users.

That's why you have DJs which are more popular than others.

If you stop playing the guitar for a year, then pick it up and try playing something, you will feel very rusty. But give it a week of practice and most of your skill comes back.. and in 1 month you're back to your peak skill.

I guess my point is - If you go full on agentic, you'll loose some of your coding skill, but you can get it back fairly quickly if you go back to manual coding. On the flip side, you get better at using AI if you use it, so your thinking is at a higher level, but you give up understanding the low level details of how exactly the code works.

Either way you're making 'music', albeit a different kind of music.

Of course if someone wants to stop using AI completely that's a completely valid decision[0], but I somewhat feel like AI is just a tool that can be easily misused.

I constantly have to review giant PRs and I noticed that I'm handwaving them more and more often. We went from almost no commit messages to walls of text that no one reads. We're starting to become bottlenecked on reviews because code is coming out too fast.

But at the same time, these are mostly issues stemming from a lack of understanding of why some of the standards/processes existed in the first place. If a developer thinks the commits have to be written just to tick a checkbox, they won't care about making them readable.

And at the same time, I'm getting a lot of value from AI, in tasks that do not necessarily have such adverse effects:

- I can create quick tools to test something, or parse/process some data. In these instances code quality is not important and I don't really want to spend hours on developing it myself (just to feel accomplished?)

- I can research issues in our codebase by just providing a log file. It's not always gonna be accurate or correct but it often gives me a very good starting point, almost always quicker than I could've done it myself

- While I do not use AI to completely generate ticket descriptions, asking it to generate me a body containing the relevant code snippets and references allows me to focus on verifying that what I'm writing is correct and understandable.

Etc etc.

So I don't know if it's just the nature of my work, the fact that I have a different skillset, or different priorities. But it somehow feels weird to me wanting to completely abandon AI just because in some cases it can lead to frustrating consequences.

[0]: I too just started a new project where I'm forcing myself to use absolutely no AI!

It all came back every time I went back to the trenches. The question in my mind is, what do "AI-native" engineers have that they can come back to?

- During these decades, I have relinquished control several times in favor of productivity. From knowing exactly where every byte is placed in RAM and where each cycle goes, to only knowing that for the inner loops, to just knowing the machine code that the C compiler will generate, to dynamic memory and classes and indirections and cache misses, to wasteful but oh so very expressive javascript and python. I stopped writing my own engines and used Unity, Unreal Engine, Phaser, Godot...

Relinquishing control is easy if you are still truly in control of the new layer, and know where the pitfalls are. Where are new engineers going to gain that expertise?

- Regarding addiction, I've also quit smoking. After 25 years of daily cigarettes, one day 16 years ago I just stopped and never touched another one.

I won't pretend that applies to everyone, or even that I'm impervious to other addictions just because that one was so easy to shed. But harder or easier, everyone can stop problematic habits if they are clearly problematic.

- I don't know where we're going with all this AI. I would prefer it had not happened the way it is happening (IP theft, job destruction, race to the bottom, power concentration, etc). I love progress but I don't think the most important aspect of progress is how fast it happens. Speed only helps the greedy and the terminally ill.

But I'm not going to pretend it hasn't, or risk whatever is left of my professional future boycotting it in favor of a different reality, or (who knows) reject a medical treatment just because it was proposed by Opus 7. The world will live or die regardless what I do, but MY world relies on me.

I will continue trying to have enough expertise, passion and attention to detail in what I do and how I do it, that whatever level of control I have over it is as optimal as I can. From typing z80 bytes, to asking Claude to change a 5 for a 6, the above traits are what has always mattered in my experience.

I see great engineers troubleshoot everything by pasting logs into the prompt and blindly accepting the answer. Zero added value while they ctrl-c ctrl-v themselves out of a job.

For the rest I do not. I do not place AI-generated code anywhere.

You lose all control AND UNDERSTANDING.

When things go wrong it gets very messy.

I will keep doing this, I think it works well, I emjoy programming and I think it is productive.

For testimg I tend to write randomized testing, which takes a bit of design but oncr you have it, well, it os test-generatove and increases the quality of checks.

Soon we rediscovered Little’s Law. WIP was piling up and we were getting overwhelmed at the integration phase, and realized that we had got really good at starting projects but actually finishing them was a struggle. Tickets were moving fine, of course. Our rate of generating code and committing PRs was through the roof. But getting actual projects to a point where the stakeholders and customers were happy with the result was just not happening.

So now we have gone back to strict WIP limits and requiring every non-trivial project to have at least two people collaborating on it. The rate at which we are churning out code has gone back down, along with the token bill, but the logjam is clearing. Better yet, the stakeholders, who never cared about our quantitative velocity metrics in the first place, have eased off on complaining that we aren’t getting anything done.

I have not lost control.

I my most prolific project I do not review the code, but I QA test extensively.

In other projects at work, I review the code.

I prompt to simplify, I challenge implementation that solves irrelevant edge cases, resulting in much smaller PRs.

In projects where I do not work alone, I still write two line PR descriptions myself.

Dumping paragraphs of AI output into the description of a MR where I ask others to review I consider disrespectful.

---

> If you turn off your brain, and relax babysitting AIs, you’re not getting any better. You’re losing value

I'm hardly turning off my brain here.

As the author notes, the context switching and so on takes concentration and effort too.

I can say without doubt that I am more productive than ever.

I am getting better by the month, and I am not currently losing value, until the AI fully replaces both me and the author.

This is a personal time management problem. Not a tool problem.

THere is a lot of temptation to do many things at the same time. For some people this works. For others it doesn't. This is not a tool problem. You simply need to fit the tools in a way that matches your personal workflow.

I can do at most two things at the same time. A friend reads a book when Claude works on a task. We're all different.

The barrier to software development has only ever been computer access and knowledge.

With AI, it’s roughly computer and internet access.

This means we’re getting a lot of people who aren’t good at either software development or AI automation playing with both. It’s the majority of what people seem to talk about.

I don’t think this is bad, but I do think it’s making real progress in AI automated software development on teams which are good at both much less visible.

A conservative team member of mine estimated we’re working at 200x speed these days, compared to 2 years ago. And we still see ways we can improve. A parallel team is only seeing an 1.2x increase, but they are unable to modify their architecture around AI.

Some of this is shifting roles. You can have a mildly technical domain expert vibe code the frontend for a new module. The more AI automation you’ve architected for, the faster they can go and the higher quality the outcome. We’re experimenting with mixing vibe coding with specifying formal requirements to push this further.

This works well. And now you’ve cut dozens of rounds of the PM not knowing the right shape for the new software out of the process. Even if we threw the end code away, this would save us tons of time.

This is just one example.

Yep. Part of the job of a software developer is managing complexity.

It's critical because of its exponential nature: as complexity increases it becomes exponentially more difficult to successfully change, understand, or support the system.

From any given position, AI seems to be the right next single move. When things are simple, it's fast and smooth. When things are complex, sota models can handle it, accounting for the details of dozens or hundreds of things at a time.

The problem is, without strict guidance, it tends to add complexity. If you let it, it builds systems at a rate and in a manner that a human cannot understand it. The best next single move leads to a poor overall position.

Then you have no choice but to keep using AI, and to keep upgrading to the next, more powerful model.

Or retreat.

It's pretty clear to me that, as a whole, we don't know yet how to develop with AI. People are mistaking token output or short-term progress for real progress. They people who are going to be successful with AI-assisted development are going to be the ones who can maintain control of their systems while they do it, and the key skill will be complexity management.

If you cannot read it as the author, what hope do I have to read and make sense of the wall of text which doesn’t seem to describe what I actually need to start reviewing.

I really really encourage everyone to write their own descriptions for PRs. If you cannot succinctly describe it in a way another human understands then you don’t understand your own change and you should withdraw your request.

I think there might be (dare I say) a middle ground to get the productivity of the llm, esp as we evolve them, while still maintain a global and even fine-grain comprehension of a code base.

It is not a simple change, however, but a fundamental one.

Overall I think we are still living in the past and try to apply ourselves to the future. But if the ai craze is to be taken clear-headedly for what it is, it is a complete break from the von Neumann computer and all its resulting artifacts. So why should we use the same tools?

> let me tell you about this experience, and how it was turning me dumber, lazy, and a worse developer.

Though the article discusses from the point of using agents, I digress to the topic of building with AI in general.

My experience has been the exact opposite. A new idea (usually related to correctness or architecture) is discussed first with the LLM where it defaults to average Joe idiotic bullshit pushback.

This frustrates me and I abuse the LLM for being idiotic by explaining the how. This results in a more refined and concrete form of the abstraction leading me to even more insights.

The LLM remains an idiot. But a useful idiot nonetheless.

It baffles me that someone who prefers TDD can't just stick to writing the tests and then using them to validate and constrain the AI's output. Only making a small number of changes per PR is easy to enforce as well. The problem here doesn't seem to be using AI, it seems to be a lack of discipline in how you use AI.

>There were tasks I could have done in 20 minutes easily, that took 5 minutes of an AI agent, and then 2 days for me to review.

This shouldn't happen at all if you constrain the problem. I've seen it happen many times when you just YOLO a vague spec and just keep prodding it to continue without paying attention to what is being done. If you have a well defined task that should only take 20 minutes of your time then it should only take 10 minutes to review the small amount of output. If the PR is 1000 lines changed then something went wrong, throw it away and rewrite your specification.

productive people who feel alienated by AI and want to just opt out.

https://www.gutenberg.org/ebooks/11231

Just having that mental paradigm will lead to more correct uses, and avoid a lot of the most ones most likely to lead one down the primrose path. This cultural moment of SF AI psychosis will pass, but we'll be stuck with the resulting code for a long time.

Also, +1 to the folks saying that code is a liability. More is almost never better. Aim for succinctness and brevity. That cuts directly against the thrust of LLMs' tendency, but will give you plenty to curate.

On the one hand, a majority of developers oppose mass data collection and privacy-invasive technologies like ad tech. Many of those same developers also oppose crypto projects because of their energy consumption and environmental impact.

And yet, those same developers have embraced frontier language models--technologies that have simultaneously scraped vast amounts of the world's data, raise serious privacy concerns, and consume massive amounts of energy.

A machine that does not learn from the feedback. The only thing these LLMs know about after the initial training is what makes it into the context. Imagine if someone couldn't learn anything new without someone else explicitly telling it what was important to the task every single time? You wouldn't call them a junior engineer.

This means you need more practice to master the tools.

I'm not saying every PR will be perfect but if you get unwanted slop and you care deeply about that then you need to up your prompting game.

I generally don't use it for code - I've been writing code for 30 years, so I'm pretty quick, and it's more productive when I look at total time for me to just write things myself.

But not always.

For example, I was recently writing some firmware in MicroPython and needed to implement Bluetooth to control the device.

It's been years since I've written python, since before asyncio. I'd never written MicroPython and didn't really know anything about Bluetooth or BLE.

The AI taught me about how to do it, taught me about how asyncio works (I'm deeply familiar with the model, just not in python), taught me what how Bluetooth works, what GATT characteristics are and gave me some example code that I then took and rewrote to fit my architecture.

I tried to do the normal "find a tutorial on the internet thing" first, but it's filled with worthless AI slop, ads and examples that are so trivial they're only clickbait. It's incredibly frustrating.

Meanwhile, the LLM was succinct, helpful and able to iteratively answer each question I had as learned and got deeper into it, with links to source references I could use to verify the information.

So, basically, a better search engine.

I've been writing code professionally since the 90s, and struggled through learning new things more times than I can count. This was by far the best experience I've had. It was where LLMs really shine.

But, that being said, at the end of the day I wrote the actual code myself and structured it to fit in with the rest of the architecture for the firmware.

I understood everything and when I had to pass it off to another engineer, I was able to describe everything, how it works, and why it's structured the way it is.

"I see luddites"

Resistance is futile, you will never code faster than AI, with less bugs, more optimized, with more features, in 200 languages, for a dozen of platforms, desktop, mobile, web, responsive, embedded, a thousand times cheaper than you, in your invisible niche market share (they already found you), not gonna happen, and then you'll cry in a corner that you were laid off, or your business got steamrolled by a new competitor selling slop that nobody understands and nobody will fix either, and you will never accept how people fall for this delusional mania if the beauty of art is in being hand made character by character in a punch card

Slop yourself or get left behind

We’re using an online, undeterministic, black-box middleman to generate our code. It’s 100% Trust me bro. No proof, no scrutiny, no guarantees.

I remember my parents saying "be careful with the bad guys who mow your lawn, fix your appliances, and drive all you kids to school. those are not people who are helping your family get things done more efficiently, those are drugs"

seriously what are we doing with this "AI is a drug" metaphor? Drugs aren't reading logs for me and writing unit tests?

> Once you start using AI coding agents, things spiral out of control very quickly.

no? maybe AI is not for you?

> Except you don’t write the code anymore, you just ask AI to do it, and half of the purpose of TDD (not biasing the tests by how you’ve implemented the code) is gone. But you feel you go so fast that you start not caring.

do you use code review tools? Did someone tell you to stop using them? Review your LLM's code, leave comments, tell your LLM to address the comments. Have you ever worked in management /architecture? Tech managers do this all day long before we had LLMs, it's not new. Except your workers are the smartest junior programmers you've ever had (and that is where AI is a problem, for sure - I worry for junior programmers today).

> It would not be so bad if things had stopped there, but that’s not how most human brains work. If you like something and you can have 2x, you’ll have it. I started pasting the whole Jira description of a ticket, and let AI implement it for me. Yay!! So powerful!!

totally! Have you not already implemented 10000 issues on your own and it's not boring yet? time for issue 10001 then? Whatever floats your boat...

> Control is an illusion. Other folks I’ve discussed this with agree that they don’t know 100% of what the code they are pushing to production actually does. I bet we not even 20%. Fucking scary.

sure have you ever accepted PRs from other people? ones that took a lot of work to read and understand? How is that different? except their code would actually break all the time if you didnt read it and now it's your bug to deal with because they're gone.

> Because if it was just about approving PRs that are perfectly written, it would be good. But for each PR I had to look at the code, the tests, the style, give feedback, switch to something else, go back again, push, see if I have any PR comments,

How do you think large projects get built? all by just one person so nobody has to review anyone else's code? (and even if you are - you still should be putting your own code up for review, reviewing it, and giving yourself comments)

> if CI passed, wait, now the linter complains.

why isn't the linting part of your CI ?

> The description is for others. Because there has to be a description, right? AI is so fucking good as sounding professional, that you relax and let it be. It’s not that I actually thought the code was good, but you get convinced over time, you get lazy, you become complacent. You stop questioning, and start accepting as good some code you would have never accepted, just because you cannot tell why it’s bad. You have lost control.

Just seems like a lot of issues here, so sure, dont use AI it's clearly not your thing. i do not let one line of code I dont think is "good" get committed, period

> Most AI developers will deny it, but deep inside, they know it’s true. Just don’t want to face it.

but dont assign my feelings. speak for yourself. I'm doing this shit for 40 years and you might find it gets a little tedious and repetitive after awhile

> Say no to drugs. Kind of a metaphor, but not quite.

it's a 100% totally wrong metaphor and it needs to die

I think AI is even worse than drugs. On the one hand, it has some elements of a slot machine (when you have irregular success with the process, it creates a more powerful cycle of dopamine release on success). On the other hand, it makes work easier, and humans are just wired to choose the easier road. It's like walking or driving a car when you have to travel 10 miles each day. I don't see how, on a civilization scale, AI won't transform our civilization for better or worse.