A Research Preview of Codex
openai.com
openai.com
We’ve long used local agents like Cursor and Claude Code, so we didn’t expect too much. But Codex shines in a few areas:
Parallel task execution: You can batch dozens of small edits (refactors, tests, boilerplate) and run them concurrently without context juggling. It's super nice to run a bunch of tasks at the same time (something that's really hard to do in Cursor, Cline, etc.)
It kind of feels like a junior engineer on steroids, you just need to point it at a file or function, specify the change, and it scaffolds out most of a PR. You still need to do a lot of work to get it production ready, but it's as if you have an infinite number of junior engineers at your disposal now all working on different things.
Model quality is good, but hard to say it's that much better than other models. In side-by-side tests with Cursor + Gemini 2.5-pro, naming, style and logic are relatively indistinguishable, so quality meets our bar but doesn’t yet exceed it.
One issue with junior devs is that because they’re not fully autonomous, you have to spend a non trivial amount of time guiding them and reviewing their code. Even if I had easy access to a lot of them, pretty quickly that overhead would become the bottleneck.
Did you think that managing a lot of these virtual devs could get overwhelming or are they pretty autonomous?
Counter-point B: AI does not get tired, does not need space, does not need catering to their experience. AI is fine being interrupted and redirected. AI is fine spending two days on something that gets overwritten and thrown away (no morale loss).
Advancements in general AI knowledge over time will not correlate to improvements in remembering any matters as colloquial as this.
Counter-counter-point B: AI absolutely needs catering to their experience. Prompter must always learn how to phrase things so that the AI will understand them, adjust things when they get stuck in loops by removing confusing elements from the prompt, etc.
I really wish that were the case. Most of the Jr Engineers I work with have to be told the same thing multiple times, in different ways, for things to stick.
B. Yea, that's true. I used to have over 4,000 GitHub contributions a year and it dropped to 1,000 as I got older and managed people. I used to be able to work 48 hrs straight but can't as much anymore...but you still have to be there to instruct the AI agent. It can't do it all on its own.
As long as I spend less time reviewing and guiding than doing it myself it's a win for me. I don't have any fun doing these things and I'd rather yelling at a bunch of "agents". For those who enjoy doing bunch of small edits I guess it's the opposite.
I don't think it benefits me much by very politely suggesting that its approach may not work and emphasize on the good part of the idea it generates and it would be better if it could try to push in another direction for example X. Like I do when talking to junior engineers.
I usually just say, "This does not work because of ..., Y is good though, change your mind and assess X now". I can also say this when I'm not actually sure whether X works or not, without worrying about my words sending an intern / a junior engineer into days of struggling on a deadend.
I find most often times, "bugs" aren't with writing code that doesn't compile or doesn't have passing tests. The "bugs" come from not understanding the requirements and what it is you're building.
I'm not entirely sure AI will help this at all. People are generally bad at describing software and how they want it to work. They are inaccurate there or entirely omit things in the requirements.
Yes, though, it would be overwhelming to manage a bunch of AI agents. Context switching and redirecting, guiding, will be very difficult and not everyone's cup of tea.
If argue this isn't really a result of AI though. Many people are already in this boat today. The industry is set up in this way with contractors and outsourced devs that are at a junior level...because it's the attraction of cheap labor. Many businesses are attracted to this beyond programming. One of the questions is going to be, is the cost per token economics cheaper? So long as it's cheaper, AI coding agents will have a future. If it proves to not be cheaper (and this could take years to prove out), then I don't think it'll be as popular. I think people will need to go back to the drawing board on how we use AI agents or use AI for other purposes (like training, education, developer onboarding, code reviews, debugging, etc.)
It also seems very telling they have not mentioned o4-high benchmarks at all. o4-mini exists, so logically there is an o4 full model right?
> It kind of feels like a junior engineer on steroids, you just need to point it at a file or function, specify the change, and it scaffolds out most of a PR. You still need to do a lot of work to get it production ready, but it's as if you have an infinite number of junior engineers at your disposal now all working on different things.
What's the benefit of this? It sounds like it's just a gimmick for the "AI will replace programmers" headlines. In reality, LLMs complete their tasks within seconds, and the time consuming part is specifying the tasks and then reviewing and correcting them. What is the point of parallelizing the fastest part of the process?
So the benefit is really that during this "down" time, you can do multiple useful things in parallel. Previously, our engineers were waiting on the Cursor agent to finish, but the parallelization means you're explicitly turning your brain off of one task and moving on to a different task.
It's going to take a while before companies catch up here and realize the "hidden costs" of AI. In a few years I think the narrative will shift to how efficient one is at using AI.
I can only imagine someone somewhere wanted to know "how much does it cost for a line of code?" And for decades no one could answer this. Frustrated with this, the neanderthal business man now turns to AI and is in ignorant bliss with the new found fire about to burn down his house.
When I was a kid, that worked with Nintendo games sure...but I like to think I've matured beyond that...but I haven't read every little thing returned by the LLM in Roo Code myself, so maybe it's human nature.
If you don't mind, what were the strengths and limitations of Claude Code compared to Codex? You mentioned parallel task execution being a standout feature for Codex - was this a particular pain point with Claude Code? Any other insights on how Claude Code performed for your team would be valuable. We are pleased with Claude Code at the moment and were a bit underwhelmed by comparable Codex CLI tool OAI released earlier this month.
rinse and repeat once task done, update #1 and cycle again. Add in another CC window if need more tasks concurrently.
downside is cost but if not an issue, it's great for getting stuff done across distributed teams..
My kid recently graduated from a very good school with a degree in computer science and what she's told me about the job market is scary. It seems that, relatively speaking, there's a lot of postings for senior engineers and very little for new grads.
My employer has hired recently and the flood of resumes after posting for a relatively low level position was nuts. There was just no hope of giving each candidate a fair chance and that really sucks.
My kid's classmates who did find work did it mostly through personal connections.
It's probably over for these folks.
There will likely(?, hopefully?) be new adjacent gradients for people to climb.
In any case, I would worry more about your own job prospects. It's coming for everyone.
I was running a quick errand between engineering meetings and saw the first few lines about hiring juniors, and I wrote a couple of comments about how I feel about all of this.
I'm not always guilty of skimming, but today I was.
Unfortunately this is not how companies think. I read somewhere more than 20 years ago about outsourcing and manufacturing offshoring. The author basically asked the same: if we move out the so-called low-end jobs, where do we think we will get the senior engineers? Yet companies continued offshoring, and the western lost talent and know-how, while watching our competitor you-know-who become the world leader in increasingly more industries.
Prior to 2022 they'd fly out the entire crew from the US and all the workers would be American and Canadian. Union, highly paid. Now they're using local (non-American) labor.
Amazon and Apple taught the foreign talent how to do grip work so they didn't have to hire expensive American workers anymore.
There are far fewer productions happening domestically within the US now. The numbers are 30% of what they once were.
I wonder if that's related
As you say, happens all the time. Also doesn’t make sense because so few people are buying individual stocks anyway. Goal should be to consistently outperform over the long term. Wall street tends to be very myopic.
Thinking long term is a hard concept for the bean counters at these tech companies i guess…
But this does not happen in industry verticals that are protected by regulation (banks) or national interest (Boring).
They'll probably just need to learn for longer and if companies ever get so desperate for senior engineers then just take the most able/experienced junior/mid level dev.
But I'd argue before they do that if companies can't find skilled labour domestically they should consider bringing skilled workers from abroad. There are literally hundreds of millions of Indians who got connected to the internet over the last decade. There's no reason a company should struggle to find senior engineers.
Money number must always go up. Hiring people costs money. "Oh hey I just read this article, sez you can have A.I. code your stuff, for pennies?"
I think some people are betting on the fact that AI can replace junior devs in 2-5 years and seniors in 10-20, when the old ones are largely gone. But that's sort of beside the point as far as most corporate decision-making.
Top-tier engineers who integrate a deep understanding of business and user needs into technical design will likely be safe until we get full-fledged AGI.
2) AIs are demonstrably much, much worse at debugging code than writing fresh code
Ex: "Oh, I see the problem! Let me fix that" -> proceeds to create a new bug while not fixing the old one
Case 1: you keep training engineers.
Case 1.1: AGI soon, you don't need juniors or seniors besides a very few. You cost yourself a ton of money that competitors can reinvest into R&D, use to undercut your prices, or return to keep their investors happy.
Case 1.2: No AGI. Wages rise, a lot. You must remain in line with that to avoid losing those engineers you trained.
Case 2: You quit training juniors and let AI do the work.
Case 2.1: AGI soon, you have saved yourself a bundle of cash and remain mostly in in line with the market.
Case 2.2: no AGI, you are in the same bidding war for talent as everyone else, the same place you'd have been were you to have spent all that cash to train engineers. You now have a juicier balance sheet with which to enter this bidding war.
The only way out of this, you can probably see, is some sort of external co-ordination, as is the case with most of these situations. The high-EV move is to quit training juniors, by a mile, independently of whether AI can replace senior devs in a decade.
Senior engineers are already very well paid. Wages rising a lot from where they already are, while companies compete for a few people, and those who can’t afford it need to lean on AI or wait 10+ years for someone to develop with equivalent expertise… all of this sounds bad for the industry. It’s only good for the few senior engineers that are about to retire, and the few who went out of their way to not use AI and acquire actual skills.
No you don't. Most engineers are shy, conflict-averse, and hate change. You can keep underpaying them and most of them will stay.
You get high EV because everyone else in your market voluntarily slowing down is a gift-wrapped miracle for you.
(Even in an AGI-soon case - you spent a bit more (let's be serious here, we're not talking about spending our entire bankroll on 18months of new hires here) in short term to get ahead, then you shift people around or lay them off. Your competitors invested that money into R&D? What does that even mean if it didn't involve hiring and AGI happens soon anyway?)
----
(Case 3: AGI soon, you don't need yourself anymore - it's hard to imagine a sufficiently advanced "AGI" that someone only replaces software devs but leaves the structure, management, and MBA-trappings of modern exchange and businesses alone.)
You lack imagination. You can eg just charge juniors for the training.
Either directly (which won't really work, because juniors almost by definition don't have a lot of money), or via a bond that they have to pay back iff they jump ship before a set number of years.
Have a look at how airlines and airline pilots pay for their expensive education.
You can get around most of your objections by using a model with open weights that you run on-premises.
I think instead we should focus on getting rid of managers and product owners.
E.g.:
If we have 10 PMs and 90 devs today, that could be hypothetically be replace by 8 PM+Dev, 20 specialized devs, and 2 specialized PMs in the future.
So you end up with some choices:
* do you move at the same speed, with fewer people?
* do you try to move faster, with less of a reduction in people? this could be trickier than it sounds because if the frequency of changes increases the frequency of unintended consequences likely does too, so your team will have to spend time reacting to that
I think the companies that win will be the second batch. It's what happens today, basically, but today you have to convince VCs or the public market to give you a bunch of more money to hire to 10x the team size. Getting a (one-off?) chance to do that through tooling improvements is a big gift, wasting it on reducing costs instead of increasing growth could be risky.
Nah, you have to put them in hamster wheels so they keep generating steady value for the shareholders, and put those in open plan offices so they get too mentally exhausted and distracted to try and change things. Throw in free cheese during good economy to keep them happy, but that's strictly optional.
Who says companies aren't doing that with AI (and technology in general) already?
The _instead_ was a key word in my comment. I didn’t say, or imply, they weren’t working on replacing other roles with AI.
That said, back in the early 00s there was much more of a culture of everyone is expected to be self-taught and doing real web dev probably before they even get to college, so by the time they graduate they are in reality quite senior. This was true for me and a lot of my friends, but I feel like these days there are many CS grads who haven't done a lot of applied stuff. But at the same time, to be fair, this was a way easier task in the early 00s because if you knew JS/HTML/CSS/SQL, C++ and maybe some .NET language that was pretty much it you could do everything (there were virtually no frameworks), now there are thousands of frameworks and languages and ecosystems and you could spend 5+ years learning any one of them. It is no longer possible for one person to learn all of tech, people are much more specialized these days.
But I agree that eventually someone is going to have to start hiring juniors again or there will be no seniors.
To contrast, CH and GER are known to have very robust and regulated apprenticeship programs. Meaning you start working at a much earlier age (16) and go to vocational school at the same time for about 4 years. This path is then supported with all kinds of educational stepping stones later down the line.
There are many software developers who went that route in CH for example, starting with an application development apprenticeship, then getting to technical college in their mid 20's and so on.
I think this model has a lot of advantages. University is for kids who like school and the academic approach to learning. Apprenticeships plus further education or an autodidactic path then casts a much broader net, where you learn practical skills much earlier.
There are several advantages and disadvantages of both paths. In summary I think the academic path provides deeper CS knowledge which can be a force multiplier. The apprenticeship path leads to earlier high productivity and pragmatism.
My opinion is that in combination, both being strongly supported paths, creates more opportunities for people and strengthens the economy as a whole.
Vocational training focusing on immediate fit for the market is great for companies that want to extract maximal immediate value from labour for minimal cost, but longer term is not good for engineers themselves.
Eg a company like Google (or similar) could probably offer you better on the job vocational training than going to uni would do to teach anyone programming.
GER isn’t even a valid country code so it compounds the confusion, if we are going to make a fake three-letter country code for Germany derived from English, why not do the same for significantly more obscure Switzerland?
The best junior I've hired was a big contributor to an open source library we were starting to use.
I think there's still lots of opportunity for honing your skill, and showing it off, outside of schools.
From my experience no one cares. You're lucky if recruiter even looks at your CV, not to mention your GitHub profile.
If your next hire is only going to stay for 1-2 years, it doesn’t make sense to hire a junior team member and invest in their growth.
As a job hopper myself, I can’t fault others for doing it though. I never hopped for the money. I just got bored or felt isolated in my role. But a nice consequence is that my salary actually appreciably increased, as opposed to colleagues/friends who stuck with the same company.
I think anything less than that, and you haven't had time to really learn an ecosystem, and more importantly you might not have had a chance to live with the consequences of your technical decisions (i.e. supporting something in prod that you built).
I know plenty of people who started somewhere, left for a while, and then came back and ended up in a position higher than they would have gotten if they had stayed put and tried to climb internally.
And yes agreed that moving around will 100% grow your comp faster than staying put (in most cases).
Like right now I've been at current co for 3 years. At the start I was getting decent raises after big projects. I now have increasingly more responsibility, I'm doing hiring, I'm doing mentorship, I'm doing huge feature work, I have to waste half my time talking to the braindead stakeholders. And what do I get for that? Absolutely jackshit, I'm getting paid the same I was when I had a quarter of the responsibility and work, yet the company is boasting about making ever more money as they lay off entire teams of people.
Why on earth would I be loyal at this point, it's clear they don't give the slightest inkling of a shit about me or anyone else who does have "Head of" or "Chief" prepended to their title.
You get what you optimize for, really.
2020-2022 was not "normal hiring", it was much higher than normal.
Even the large, conservative, boring enterprise where I work was hiring about 5 new developers a week in those years.. I know because I did the "Welcome to our department" technical onboarding workshop and I saw everyone coming in the door every week.
Before 2020 I ran that workshop once a month or so? Probably 10 times a year at most.
So of course then the layoffs came when ZIRP ended, as everyone had to adjust back to real world conditions and grapple with the fact that the team sizes they had now were not sustainable (or necessary?) when free money ended.
Couple that with "efficiencies" from AI and generally more conservative scope and ambition by a lot of companies in these uncertain times, and the market looks drastically worse than it did in 2020-2022.
But if you compare the market today to 2018 (for example) instead, it's still worse but definitely not as worse.
Lastly, hiring is also very broken today due to AI, with candidates using it to flood companies with applications and then companies using AI to auto-filter that incoming flood (and likely losing out on good candidates in the process) simply because they can't handle the volume.
I have a lot of sympathy for folks coming right out of university trying to land their first good job. I'm sure it's the toughest it's been in a generation...
Juniors cuz they're cheap and motivated (and potentially grow into very good team members!), seniors to handle the trickier stuff. Mid-level people don't get picked up cuz there's enough junior pipeline that the cost-benefit doesn't work out.
Thing is these companies tend to have, say, college pipelines and the like for juniors. Internships and the like. It would be really painful to not have internships lined up at "your local startup" in this day and age.
My impression is that a lot of the junior dev pipeline is in smaller places that don't show up in job boards in the same way as the rest of it. Maybe that's dried up too, but I have my doubts.
You still need somebody to work the robot, even if the robot is "doing the coding"!
The problem is getting hired. If seniors are having problems getting callbacks for interviews right now, then a young kid with a thin resume isn't going to get hired, no matter how senior their skills are in reality.
This is entirely a combination of two things: The quality of grads coming out of college/university, and pressures coming from the market. Colleges have been pushing through entirely unqualified students, some even language illiterate, into the market place and what we're seeing is a response to that. Now couple that with the pressures that companies are facing, and you can see why none of them want to even take on the risk of training and up-skilling someone just so they can find the actual good employees which are a small percentage.
Of course, in my company's particular country and context, government regulations make it impossible to fire someone and there is huge pressure to keep-up DEI quotas despite no actual good DEI candidates being available, and we have a mess. Day to day is glorified baby-sitting people not-knowing what to do, dealing with their "feelings" (usually feelings of inadequacy and sometimes snobbish entitlement) and still trying complete a project at the same time.
I mean even compared to just not hiring any juniors.
Oh well, I know that it happens, saw it in 2010 with "cloud" when it was basically still just EC2,S3,RDS, and whatever the not-haproxy-but-haproxy load balancer was called, ELB. Job poatings asking for half a decade or more of experience. I always get the feeling there's some requirement they post jobs public but only hire internal, but I have no way to prove that; I have heard others say this, though.
They will sell them as 'junior SE' on the first 2 projects, 'medior SE' on the next and then they'll be 'senior SE' within 18 months tops of leaving bootcamp.
The projects by these shops are nearly always troubled, requiring many (customer paid in change requests) reworks before (if ever) getting in production.
They seldom are hired to do projects for a client twice, but it's a lemom's market and there's plenty of fish in the sea.
So what happens with these shops is that their SE's will rely even more than average on AI assistants, and deliver a v0.1 faster, but with 10x more code and holes than before, taking even longer to get in production but given lemons and fish have not changed they'll still hire, now even cheaper out of 'prompt engineering bootcamp'
We still have a large funnel of interns that end up becoming junior devs, and then progressing normally. I don't know the exact ratio of interns that end up actually getting hired as full-time employees, it's definitely low, but I think this is more of a function of most of them not actually being any good.
High salaries in programming has attracted a lot of people who have no passion for the craft. To get good you have to have that and it takes years of either school or passionate autodidactic learning or both.
But also, I think this underestimates significantly what junior engineers do. Junior engineers are people who have spent 4 to 6 years receiving a specialised education in a university - and they normally need to be already good at school math. All they lack is experience applying this education on a job - but they are professionals - educated, proactive and mostly smart.
The market is tough indeed, and as much it is tough for a senior engineer like myself, I don't envy the current cohort of fresh grads. It being tough is only tangentially related to the AI though. Main factor is the general economic slowdown, with AI contributing by distracting already scarce investment from non-AI companies and producing a lot of uncertainty in how many and what employees companies will need in the future. Their current capabilities are nowhere near to having a real economic impact.
Wish your kid and you a lot of patience, grit and luck.
Without being overly pessimistic, this interpretation is extremely generous.
What I've mostly seen when it isn't the case is an employer who hasn't let them fly, but simply bombarded them with work nobody wants to do. They become cynical, and it is understandable.
I have mentored junior developers and found it to be a rewarding part of the job. My colleagues mostly ignore juniors, provide no real guidance, couldn't care less. I see this attitude from others in the comments here, relieved they don't have to face that human interaction anymore. There are too many antisocial weirdos in this industry.
Without a strong moral and cultural foundation the AGI paradigm will be a dystopia. Humans obsolete across all industries.
That's really awesome. I hope my daughter finds a job somewhere that values professional development. I'd hate for her to quit the industry before she sees just how interesting and rewarding it can be.
I didn't have many mentors when starting out, but the ones I had were so unbelievably helpful both professionally and personally. If I didn't have their advice and encouragement, I don't think I'd still be doing what I'm doing.
Can totally relate. Unfortunately the trend for all-senior teams and companies has started long before ChatGPT, so the opportunities have been quite scarce, at least in a professional environment.
Here’s a possible out: Senior engineers stop working huge corporations and use these tools to start their own businesses. (Given today’s hiring situation, this may not even be a choice.) As the business grows, hire junior developers as apprentices to handle day to day tasks while senior engineer works on bigger picture stuff. Junior engineer grows into a senior engineer who eventually uses AI to start their own business. This is a very abbreviated version of what I hope I can do, at least.
Your solution cannot work at scale, because if the small companies you propose succeed, then they will become corporations, which, as you say, cannot be depended upon to do the right thing.
All the same principles apply as before: smart, driven, high ownership engineers make a huge difference to a company's success, and I find that the trend is even stronger now than before because of all the tools that these early career engineers have access to. Many of the folks we've hired have been able to spin up on our codebase much faster than in the past.
We're mainly helping them develop taste for what good code / good practices look like.
That's really great to hear.
Your experience that a new engineer equipped with modern tools is more effective and productive than in the past is important to highlight. It makes total sense.
There’s still quite a bit of a gap in terms of trust.
It's not long ago when the correction of the tech job market started, because it got blown up during and after covid. The geopolitical situation is very unstable.
I also think there is way too much FUD around AI, including coding assistants, than necessary. Typically coming either from people who want to sell it or want to get in on the hype.
Things are shifting and moving, which creates uncertainty. But it also opens new doors. Maybe it's a time for risk takers, the curious, the daring. Small businesses and new kinds of services might rise from this, like web development came out of the internet revolution. To me, it seems like things are opening up and not closing down.
Besides that, I bet there are more people today who write, read or otherwise deal directly with assembly code than ever before, even though we had higher level languages for many decades.
As for the job market specifically: SWE and CS (adjacent) jobs are still among the fastest growing, coming up in all kinds of lists.
Today startups mostly wrap LLMs as this is what VCs expect. Larger companies have smaller IT budgets than before (adjusted for inflation). This is the real problem that causes the jobs shortage.
It's not going to get any easier in next next few years, I think. Till the point when fresh grad using AI can make something valuable. After that it will be period when anybody can just ask AI to do something and it will find soft in its library or write from scratch. In long terms, 10 years may be, humanity probably will not need this many developers. There will be split like in games industry: tools/libs developers and product devs/artists/designers. With the majority in second category.
Young people are already doing that, but a lot of what they produce is what you expect from people who have no prior experience in designing and testing software for production environments.
But I also wonder (I'm thinking out loud here, so pardon the raw unfiltered thoughts), if being a junior today is unrecognizable.
Like for example, that whatever a "junior" will be now, will have to get better at thinking at a higher level, rather than the minute that we did as juniors (like design patterns and all that stuff).
So maybe the levels of abstraction change?
This problem might be new to CS, but has happened to other engineers, notably to MechE in the 90's, ChemE in 80's, Aerospace in 70's, etc... due to rapid pace of automation and product commoditization.
The senior jobs will disappear too, or offshored to a developing country: Exxon (India 152 - 78 US) https://jobs.exxonmobil.com/ Chevron (India 159 - 4 US) https://careers.chevron.com/search-jobs
Golden age of software development will be over soon? Probably, for humans. How cool is it, the most enthusiastic part will be replaced first.
2015-2022 was peak, downhill from there and it doesn't look like it'll recover.
a lot of junior eng tasks don't really help you become a senior engineer. someone needs to make a form and a backend API for it to talk to, because it's a business need. but doing 50 of those doesn't really impart a lot of wisdom
same with writing tests. you'll probably get faster at writing tests, but that's about it. knowing that you need the tests, and what kinds of things might go wrong, is the senior engineer skill
with the LLMs current ability to help people research a topic, and their growing ability to write functioning code, my hunch is that people with the time to spare can learn senior engineer skills while bypassing being a junior engineer
convincing management of that is another story, though. if you can't afford to do unpaid self-directed study, it's probably going to be a bumpy road until industry figures out how to not eat the seed corn
That's been the case for most of the last 15 years in my experience. You have to follow local job markets, get in through an internship, or walk in at local companies and ask. Applying en mass can also help, and so does having some code on GitHub to show off.
As everything is so new and different at this stage we are in a state of discovery which requires more senior skills to work out the lay of the land.
As we progress, create new procedures, processes, and practices, particularly guardrails then hiring new juniors will become the focus.
I know this isn't what you want to hear, but what makes you think senior engineers will be in short supply in "the future"?
I'm not even a developer (anymore, I was in the past), I'm a product manager, and I'm pretty sure I can see the point in a few years where not just developers but people like me get disintermediated. My customers have a semi-reasonable grasp of what they're looking for, and they can speak. In a few years -- ten at the absolute most -- my customers will say to an AI, "I need an application that does XYZ" and the AI will reply, "Are you sure about that? Most people who say they need XYZ end up using an app that does WXY." My (former) users will reply, "Let's try it my way and see what happens." And the AI will say, "Okay, here are three popular UI styles, which do you prefer?" etc. etc.
We're headed for Interesting Times.
I would say: ten years ago there was a huge shortage of engineers. Today, there is still lots of coding to be done, but everyone is writing code much faster and driven people learn to code to solve their own problems
Part of the reason it was so easy to get hired as a junior ten years ago was because there was so much to do. There will still be demand for engineers for a little while and then it's possible we will all be creating fairly bespoke apps and I'm not sure old me would call what future me does "programming".
On top of that, you need to be really sharp at leetcode for any large-ish company.
I find the "ai tools are junior engineers" narrative flawed, but it has any way accelerated the higher and higher expectations for a junior.
- greybeard who is trying his hardest to keep up
It stops juniors investing their life/time/energy in a field that is shrinking and that will increasingly "not be worth it" w.r.t effort put in given their longer time horizon. This is how capitalism when working correctly can obsolete jobs somewhat charitably - it does it by closing the door on entry level jobs ideally when people have little to lose and haven't yet invested a lot of their life into it. For example they may still be young enough to re-train; or may be dismayed from entering the field due to disruption/chatter and so do something more appropriate in the new world.
Being hired in a sinking and increasingly more competitive field may actually be considered a "winner's curse" outcome, in that you will be in a industry highly competitive that is slowly sinking and is stressful with low opportunities for pay rises compared to other industries/skill sets - this is definitely playing your career in "hard mode". Most of all you will feel your skills, and value is useless relatively to people who got into more jobs with more scarcity playing life in "easy mode" with less stress and anxiety. In a few years time people getting into other fields may feel they "dodged a bullet" comparing themselves to others that did.
Being able to pivot while you are still young and ageism isn't a barrier yet is definitely something to consider remembering careers these days are multi-decades long. I feel for your kid now, and I do for mine, but I would rather than try something different in their 20's vs say their 40's when they have a mortgage, a family to feed, and/or other commitments and ageism makes it harder to pivot/re-train into another career. I don't wish my kids to feel the anxiety I and many people I know are feeling later in life especially for a career that requires constant effort to maintain and keep relevant in. I'm not recommending my kids learn what I do at all for example.
I am utterly perplexed with the current situation on the job market, which seems to be a global phenomenon that is not constrained to a particular country or region. Late last year, I was hiring for two junior software engineering positions and (through an external recruitment partner) we received over 400 job applications for two junior positions. We, however, scrambled to narrow the number of candidates down to ten, out of which eight turned out to be lemons and two ended up being exceptionally good. 390 other applicants ended up being pure white noise.
Colleagues in a neighbouring business unit reported receiving over 600 submissions for a single position.
I have approached a few headhunters in the last couple of months with informal questions about what has been happening. They are under constant duress, receiving hundreds upon hundreds of applications for pretty much any position. The feedback is that when most people see a job ad, they put their resume through GenAI and submit whatever garbage comes out of it without even looking at the output. The vast majority of people can't even be bothered to write a simple cover letter, which could have been used as a shibboleth for the hiring manager / recruiter: «I am an intelligent human being, and I am real».
Naturally, the headhunters have responded with GenAI-assisted tools to sift through piles of putrid trash. The side effect is that such good, qualified applicants do not usually get a chance to get screened in.
The situation does not seem to be changing, and the only way out seems to be applying through a professional network or connections. People abusing GenAI are hurting themselves (ironically, GenAI has become pretty good at recognising GenAI-generated content), and they are also hurting pretty much everyone else in the process, and they do not care.
If this goes on for longer a wework for applicants might be an opportunity.
In fact not being bottle necked by senior engineers or not having to drawe the luck of a bad senior engineer/mentor, there will be a new stars of Junior engineers.
What you should be worried is Senior Engineers who hate AI
Hasn't this been a common refrain whenever someone found a way to automate any menial task in any job?
Cynical answer for the immediate future: maybe from the pool of existing seniors and principals that have all been stuck doing a faux-management job onboarding, mentoring and managing the juniors who were the only ones actually writing any code in many tech companies? It's a reversal of a trend, for sure: my feeling about the market until the last year or two, was that there's hardly any job for seniors or above, that isn't just management without the title and its privileges.
Past the immediate future, if we end up replacing juniors with LLMs, then the next cohort of "seniors" will need to come from some kind of vocational training.
This is a very valid concern that predates AI by decades. AI just makes it worse. How will we raise the next generation of experts when there is no entry level of anything? We have either outsourced or automated everything below mid career level.
This is an area that I agree with some on the nationalist right — at least about the diagnosis, but not about the cure. If we continue down this road we end up with abandoned generations struggling to pay bills beneath an entrenched gerontocracy. If we do crack any kind of real age reversing life extension this could get really dystopian, like bad cyberpunk movie stuff, where you have generations of the impoverished beneath a pickled elite that never dies and owns everything.
etc.
This would not save me time. It would be paying it forward. And I cannot do this.
This is also part of a recent update to Zed. I typically use Zed with my own Claude API key.
That said, it might be possible to tell each agent to create a branch and do work there? I haven't tried that.
I haven't seen anything about Zed using containers, but again you might be able to tell each agent to use some container tooling you have in place since it can run commands if you give it permission.
If you want one idiot's perspective, please hyper-focus on model quality. The barrier right now is not tooling, it's the fact that models are not good enough for a large amount of work. More importantly, they're still closer to interns than junior devs: you must give them a ton of guidance, constant feedback, and a very stern eye for them to do even pretty simple tasks.
I'd like to see something with an o1-preview/pro level of quality that isn't insanely expensive, particularly since a lot of programming isn't about syntax (which most SotA modls have down pat) but about understanding the underlying concepts, an area in which they remain weak.
Atp I really don't care if the tooling sucks. Just give me really, really good mdoels that don't cost a kidney.
Does this mean people will be less incentivized to contribute to open source as time goes by?
P.S., I think the current trend is a wakeup call to us software engineers. We thought we were doing highly creative work, but in reality we spend a lot of time doing the basic job of knowledge workers: retrieving knowledge and interpolating some basic and highly predictable variations. Unfortunately, the current AI is really good at replacing this type of work.
My optimistic view is that in long term we will have invent or expand into more interesting work, but I'm not sure how long we will have to wait. The current generation of software engineers may suffer high supply but low demand of our profession for years to come.
Most of the waking hours of most creative work have this type of drudgery. Professional painters and designers spend most of their time replicating ideas that are well fleshed-out. Musicians spend most of their time rehearsing existing compositions.
There is a point to be made that these repetitive tasks are a prerequisite to come up with creative ideas.
How much of it is brute force?
Really?
Not to say it's not impressive and good.
no it creates shit thats close enough for people who are in a rush and dont care.
ie, you need artwork for shit on temu, boom job done.
You want to make a poster for a bake sale, boom job done.
Need some free music that sounds close enough to be swifty, but not enough to get sued, great.
But as an expression of creativity, most people cant get it to do that.
Its currently slightly more configurable clipart.
Random HNer on an AI post one day later
> Its currently slightly more configurable clipart.
It's so ridiculous at this point that I can just laugh about this.
AI music is disposable generic soulless trash, even if it is technically correct, in accordance with the rules and conventions of music theory. AI generates Muzak. Totally generic and derivative.
There is no AI equivalent to Kurt Cobain, or James Brown, or Tori Amos.
This just in, the response was also to an entirely different context.
> rowanG077: Holy shit look how different the opinions are! LOL HN is so wild.
AI is never going to give the world a modern Kraftwerk or Silver Apples or Brian Eno. The best an AI "artist" can do is have the machine mimic them.
AI relies on prompting. In the hands of a skilled artist it is just another tool. In the hands of an amateur hack, it is no different than giving a drum kit to a 4 year old.
The tool is not the art. It never was.
In the same way that the synth is superstitious is banging.
Sampling, when in the hands of a legend is also spectacular, see the prodigy and a break down of the samples they used. (or any half decent hiphop band)
Then you get akon who just sped up a single sample put a beat on it and shat out some halfarsed shit.
You can do the same for photography.
People keep lowering expectations or demands on quality because things get easier and humans always prefer the easy option.
If you extrapolate and generalize further... what is at risk is any task that involves taking information input (text, audio, images, video, etc.), and applying it to create some information output or perform some action which is useful.
That's basically the definition of work. It's not just knowledge work, it's literally any work.
For that reason all my silly little side projects are now in private repos. I dont care the chance somebody builds a business around them is slim to none. Dont think putting a license will protect you either. You'd have to know somebody is violating your license before you can even think about doing anything and that's basically impossible if it gets ripped into a private codebase and isnt obvious externally.
I'm quite conflicted on this assessment. On one hand, I was wondering if we would get better job market if there were not much open-sourced systems. We may have had a much slower growth, but we would see our growth last for a lot more years, which mean we may enjoy our profession until our retirement and more. On the other hand, open source did create large cakes, right? Like the "big data" market, the ML market, the distributed system market, and etc. Like the millions of data scientists who could barely use Pandas and scipy, or hundreds of thousands of ML engineers who couldn't even bother to know what semi positive definite matrix is.
Interesting times.
What harm is there to you if someone uses some of your code to build a business, as compared to not doing so? How are you worse off?
I’ve never understood this mentality. It seems very zero sum and kind of anti social. I’ve built a couple of businesses, and there’s always economic or technical precedent. I honestly don’t mind paying it forward if someone can benefit from side projects I enjoyed doing anyways.
This someone might be someone I dislike. It would cause me some mild annoyance that they benefited from my effort.
Spite is also pretty pleasurable.
If you decide to put open source out there for others to use, then sure. You chose to allow that.
If you didn't license it for that, then it shouldn't be in an LLM.
--- OPTION A - Keep your project private.
• You get five happiness points.
--- OPTION B - Make your project public.
• Other individuals may get a small number of happiness points.
• A megacorp might turn your project into a major product without compensating you and get a million happiness points.
• You get five happiness points.
----------
In either scenario, you still end up with five happiness points. If you release your code, other people may get even more happiness points than you, which isn't really fair. But you are no worse off, and you've increased humanity's total wealth of happiness points.
https://knowyourmeme.com/memes/total-happiness-in-the-world-...
Not that hobbyists don't contribute, but these models are certainly being trained on the work of salaried engineers as much as their trained on hobbyists' spare time projects.
personally, I completely stopped 2 years ago
it's the same as the stack overflow problem: the incentive to contribute tends towards zero, at which point the plagiarism machine stops improving
Once everyone uses AI. How will anything new come about?
AI could quite possibly slow down innovation and improvement.
More generally, specialty knowledge is valuable. From now on, all employees will be monitored in order to replace them.
This is going on a t-shirt.
Yes. I certainly don't intend to put any free code online until I can legally bar AI bros from using it without payment. As Mike Monteiro put it long ago, "F** you, pay me" (https://www.youtube.com/watch?v=jVkLVRt6c1U)
Or is all this a nothing-burger, since the new hires will just be commanding bots of their own, but on lower level tasks that they are qualified to supervise?
I work with Junior devs. They are Junior in that they have to be pointed in the direction they must work, and would be out of their depth in guiding the implementation of major features. But simple code changes are pretty fine.
LLMs are useful, I kind of like them as code assistants, but they are very far of being even a Junior dev in terms of actually performing work.
Preview video from Open AI: https://www.youtube.com/watch?v=hhdpnbfH6NU&t=878s
As I think about what "AI-native" or just the future of building software loos like, its interesting to me that - right now - developers are still just reading code and tests rather than looking at simulations.
While a new(ish) concept for software development, simulations could provide a wider range of outcomes and, especially for the front end, are far easier to evaluate than just code/tests alone. I'm biased because this is something I've been exploring but it really hit me over the head looking at the Codex launch materials.
You mean like automated test suites?
There's already library's for QA testing and VLM's can give critique on a series of screenshots automated by a playwright script per branch
Ambitious idea, but I like it.
https://huggingface.co/blog/smolvlm
recently both llama.cpp and ollama got better support for them too, which makes this kind of integration with local/self-hosted models now more attainable/less expensive
It made changes in TS/Next.js given just the boiletplate from create-next-app, ran `yarn dev` then opened its mini LLM browser and navigated to localhost to verify everything looked correct.
It found 1 mistake and fixed the issue then ran `yarn dev` again, opened a new browser, navigated to localhost (pointing at the original server it brought up, not the new one at another port) and confirmed the change was correct.
I was very impressed but still laughed at how it somehow backed its way into a flow the worked, but only because Next has hot-reloading.
Definitely ambitious!
https://constelisvoss.com/pages/a-computer-can-never-be-held...
I think the issue is not about humans being entirely replaced. Instead, the issue is that if AI replaces enough number of knowledge workers while there's no new or expanded market to absorb the workforce, the new balance of supply and demand will mean that many of us will have suppressed pay or worse, losing our jobs forever.
It's a crucial insight that's usually missed or eluded in discussions about automation and workforce - unless you're literally at the beginning of your career, losing your career to automation screws you over big time, forever. At best, you'll have to downsize your entire lifestyle, and that of your family, to be commensurate with your now entry-level pay. If you're halfway through the career that suddenly ended, you won't recover.
All the new jobs and markets are for the kids. Mind you, not your kids - your kids are going to be disadvantaged by their household being suddenly thrown into financial insecurity or downright poverty, and may not even get a chance to start a good career path with their peers.
That, not "anti technology sentiment", is why Luddites smashed the looms. Those were people who got rug-pulled by business decisions and thrown into poverty, along with their families and communities.
I feel like I've been thinking along similar lines recently (due to re-read this though!) but instead of "computer" am replacing it with "AI" or "Agents" these days. Same point holds true.
Reviewing has never been a panacea. It’s a best-effort at catching obvious mistakes, like a second opinion. Only with highly rigorous tests can reviewing give as high confidence as I trust another engineer or myself. Generally cadence of code output has never been a bottleneck for me, rather the opposite (if I had more time I’d write you a shorter letter).
Most importantly, writing code that is testable on meaningful boundaries is an extremely difficult and delicate art form, which ime is something you really want to get right if possible. Not saying an AI can or can’t do that, only that it’s the hardest part. An army of automated junior engineers still can’t win over the complexity beast that yolo programming causes. At some point code mutations will cause more problems as side effects than what they fix.
This resonates a lot for me, completely agreed.
This was always true. Front-End code is not really code. Most of the back-end code is just convert and moving data around. For most functionality where you need "real code" like crypto, compression, math, etc.. you use a library used by another 100k developers.
Look at the results from multi swe bench - https://multi-swe-bench.github.io/#/
swe polybench - https://amazon-science.github.io/SWE-PolyBench/
Kotlin bench - https://firebender.com/leaderboard
We also have SWE-bench Multimodal which adds a twist I haven't seen elsewhere: https://www.swebench.com/multimodal.html
There must be room for a Modal/Cloudflare/etc infrastructure company that focuses only on providing full-fledged computer environments specifically for AI with forking/snapshotting (pause/resume), screen access, human-in-the-loop support, and so forth, and it would be very lucrative. We have browser-use, etc, but they don't (yet) capture the whole flow.
Demo: https://surf.e2b.dev
I'm quietly betting that agents increase the leverage of deterministic, reproducible devbox tech (eg. Nix, lockfiles, package mirroring), and this will end up being a huge win for us human engineers too.
Is this still rolling out? I dont need the team plan too do I?
I have been using openAI products for years now and I am keen to try but I have no idea what I am doing wrong.
Here's my workflow that keeps failing: - it writes some code. It looks good a first glance - I push it to github - automated tests on github show that there's a problem - go back to codex and ask it to fix it - it does stuff. It looks good again.
Now what do I do? If I ask it to push again to github, then it will often create a pull request that doesn't include stuff from the first pull request, but it's not a pull request that stacks on top of the previous pull request, it's a pull request that stacks on top of main.
When asked to write something that called out to gpt-4.1-mini, it used openai.ChatCompletion.create (!?!!?)
I just found myself using claude to fix codex's mistakes.
If both OpenAI and Devin are falling into the same pattern then that’s a good indication there’s a fundamental problem to be solved here.
One of the things I am constantly struggling with is that the containers they use are having issues to fetch anything from the internet:
error: failed to get `anyhow` as a dependency of package `yawl-core v0.1.0 (/wor
kspace/yawl/core)`
Caused by:
download of config.json failed
Caused by:
failed to download from `https://index.crates.io/config.json`
Caused by:
[7] Could not connect to server (Failed to connect to proxy port 8080 after 30 65 ms: Could not connect to server)
Took me an few hours today to figure out how to install maven and have it download all the dependencies. Spent an hour trying to figure out why sudo apt-get update was failing, it was because I was was using sudo!
You can make arbitrary teams, like legal, make PRs. You would still have the proper owners of the project agree whether they take the PRs. Either by human review and/or by any other review process they set up.
You have two choices:
1. A developer makes a PR. They build the app and run it themselves to make sure it works, does what they intended, and nothing unexpected happens, and report on their testing in the PR. There's also a suite of automated tests. Between these you are confident that you can rubber-stamp it without touching the change yourself. But this requires them being able to run the code and intelligently think about it themselves, which AI cannot do.
2. A non-developer uses an LLM to make a PR. The change passes the tests, but you have no other validation about it because it was done by a bot that can't think about how the app actually works. As reviewer you have to pull down the change to run and validate it yourself. Now you are doing the same amount of work as before, except that instead of you telling the LLM to make the change, someone else did.
The only difference is maybe that the activation energy for doing the work was avoided. Which, fine, is non-negligible. But let's not pretend like in the latter case legal "made the change". They did the 1% upfront work of asking the LLM to find where the change goes, and left you the other 99% of actually shepherding it through. The only changes that will work for at all are, like, updating copy/strings/icons---and honestly, at my last job at least, we already let legal (and product, etc) do stuff like that. I suppose the LLM might save them having to figure out how to use Git, at least.
You might imagine there's some software out there where the automated tests are so thorough that you can trust a code change to not break anything if it passes the tests. I have personally never seen such a thing. And in practice many tests validate against strings and other small feature-level details, meaning that the kinds of code changes that other orgs are making are going to be touching the tests as well, so human verification is still required.
So they can at least check whether the change does what they wanted it to do.
For this to be useful, the code doesn't even have to be good: think of it as a more accurate way to gather requirements and make a prototype. That's useful, even if you throw away the code.
GitHub's codespaces also drive home how the customer/user only needs a browser, which even people in legal will have. No additional software required on their end.
What does that mean? Surely this should have a bit more elaboration. If you're just excluding a double digit number of tasks in the benchmark as uncompleted, that should be reflected in the scores.
codex-1 has been quite good in my experience
“Here’s an error reported to the oncall. Give a try fixing it” (Could be useful even if it fails)
Refactor this small piece I noticed while doing something else. Small-scoped stuff that likely wouldn’t get done otherwise.
I wouldn’t ask LLMs for full-features in a real codebase but these examples seem within the scope of what they might be able to accomplish end-to-end
Agent 1: write tests, make sure all the tests pass.
Agent 2: concert python script to fastapi
Agent 3: create frontend based on fastapi endpoints
I get a PR, I check code and see if it works and then merge to main. All three PR’s worked flawlessly (front end wasn’t pretty).
What about using it for AI / developing models that compete with our new overlords?
Seems like using this is just asking to get rug pulled for competing with em when they release something that competes with your thing. Am I just an old who’s crowing about nothing? It’s ok for them to tell us we own outputs we can’t use to compete with em?
Facebook has been caught in recent DOJ hearings breaking the law with how they run their business, just as one example. They claimed under oath, previously, to not be doing X, and then years later there was proof they did exactly that.
https://youtu.be/7ZzxxLqWKOE?si=_FD2gikJkSH1V96r
A companies “word” means nothing imo. None of this makes sense if i’m being honest. Unless you personally have a negotiated contract with the provider, and can somehow be certain they are doing what they claim, and can later sue for damages, all of this is just crossing your fingers and hoping for the best.
Of _course_ they are training on your shit.
Which is the same law that protects "publicly accessible shit".
In the same way that you can't legally just rip MP3s off the internet[1], or copy movies and distribute them.
[1] unless you're rich.
However for people trying to compromise your system access to your code can be a valuable asset .The worth of that could be well beyond just enterprise value of the organization , it could people’s lives or bring down critical infrastructure.
You don’t just have access to code you created and have complete control to. Organizations have vendors providing code(drivers , libraries…) with narrow licenses that prohibit sharing or leaking in anyway. So this type of leak can open you to a lot of liability.
When I'm using aider, after it make a commit what I do, I then immediately run git reset HEAD^ and then git diff (actually I use github desktop client to see the diff) to evaluate what exactly it did, and if I like it or not. Then I usually make some adjustments and only after that commit and push.
The secret sauce here seems like their new model, but I expect it to come to API at some point.
They seem to be injected fine in the "environment setup" but don't seem to be injected when running tasks against the enviornment. This consistently repros even if I delete and re-create the enviornment and archive and resubmit the task.
I don't understand why OAI puts their alpha release products under a $200 a month plan instead of just charging for tokens.
If anyone could use Codex, you’d want to price the tokens at a rate where you don’t lose money, or at least not too much money. If you price the tokens too high, then not only are people not using Codex, but customers and investors alike begin to lose faith, they no longer “feel the AGI”.
But they aren't moving nearly as fast as OpenAI. And it remains to be seen if first mover will mean anything.
Still super confusing, though!
I feel like companies working with and shipping LLMs would do well to remember that it's not just humans who get confused by this, but LLMs themselves... it makes for a painful time, sending off a request and noting that a third of the way into its reasoning that the model has gotten tow things with almost-identical names confused.
I assume one easy next step is to just run GitHub Actions in the container since everything is defined there (assuming the user set it up)
This seems imply that the software engineering as a profession has been quite mature and saturated for a while, to the point that a model can predict most of the output. Yes, yes, I know there are thousands of advanced algorithms and amazing systems in production. It's just that the market does not need millions of engineers for such advanced skills.
Unless we get yet another new domain like cloud or like internet, I'm afraid the core value of software engineers: trailblazing for new business scenarios, will continue diminishing and being marginalized by AI. As a result, we get way less demand for our job, and many of us will either take a lower pay, or lose our jobs for extended time.
Docs could make that more clear
I made one for github action but it's not as realtime and is 2 years old now: https://github.com/asadm/chota
A not open-source option this looks close to is also https://githubnext.com/projects/copilot-workspace (released April 2024, but I'm not sure it's gotten any significant updates since)
This should be possible today and surely Linus would also see this in the future.
(Im trying something)
what would be an impressive program that an agent should be able to one-shot in one go?
I can't say I am a big fan of neutering these paradigm-shifting tools according to one culture's code of ethics / way of doing business / etc.
One man's revolutionary is another's enemy combatant and all that. What if we need top-notch malware to take down the robot dogs lobbing mortars at our madmaxian compound?!
I wouldn't sweat it. According to it's developers, Codex understands 'malicious software', it has just been trained to say, "But I won't do that" when such requests are made to it. Judging from the recent past [1][2] getting LLMs to bypass such safeguards is pretty easy.
1.https://hiddenlayer.com/innovation-hub/novel-universal-bypas... 2.https://cyberpress.org/researchers-bypass-safeguards-in-17-p...
Feels like codex is for product managers to fix bugs without touching any developer resources. Then it’s insanely surprising!
But now who's going to do that work? Still engineers.
I'm very interested.
In my experience ChatGPT and Gemini are absolutely terrible at these types of things. They are constantly wrong. I know I'm not saying anything new, but I'm waiting to personally experience an LLM that does something useful with any of the code I give it.
These tools aren't useless. They're great as search engines and pointing me in the right direction. They write dumb bash scripts that save me time here and there. That's it.
And it's hilarious to me how these people present these tools. It generates a bunch of code, and then you spend all your time auditing and fixing what is expected to be wrong.
That's not the type of code I'm putting in my company's code base, and I could probably write the damn code more correctly in less time than it takes to review for expected errors.
What am I missing?
personally, i've always operated in a codebase in a way that i _need_ to understand how things work for me to be productive and make the right decisions. I operate the same way with AI. every change is carefully reviewed, if it's dumb, i make it redo it and explain why it's dumb. and if it gets caught in a loop, i reset the context and try to reframe the problem. overall, i'm definitely more productive, but if you truly want to be hands off--you're in for a very bad time. i've been there.
lastly, some codebases don't work well with AI. I was working on a problem that was a bit more novel/out there and no model could solve it. Just yapped endlessly about these complex, very potentially smart sounding solutions that did absolutely nothing. went all the way to o1-pro. the craziest part to me was the fact that across claude, deepseek and openai, they used the same specific vernacular for this particular problem which really highlights how a lot of these models are just a mish-mash of the same underlying architecture/internet data. some of these models use responses from other models for their training data, which to me is like incest. you won't get good genetical results
That you are trying to use LLMs to create giant sprawling codebase feature packed software packages that define the modern software landscape. What's being missed is that any one user might only utilize 5% of the code base on any given day. Software is written to accommodate every need every user could have in one package. Then the users just use the small slice that accommodates their specific needs.
I have now created 5 hyper narrow programs that are used daily by my company to do work. I am not a programmer and my company is not a tech company located in a tech bubble. We are a tiny company that does old school manufacturing.
To give a quick general example, Betty uses Excel to manage payroll. A list of employees, a list of wages, a list of hours worked (which she copys from the time clock software .csv that she imports to excel).
Excel is a few million LOC program and costs ~$10/mo. Betty needs maybe 2k LOC to do what she uses excel for. Something an LLM can do easily, a python GUI wrapper on an SQLite DB. And she would be blown away at how fast it is, and how it is written for her use specifically.
How software is written and how it is used will change to accommodate LLMs. We didn't design cars to drive on horse paths, we put down pavement.
But yes, I hope we get away from the giant conglomeration of everything, ESPECIALLY the reality of people doing 90% of their business inside a Google Chrome widow. Move towards the UNIX philosophy of tiny single-purpose programs.
OK, great.
> That you are trying to use LLMs to create giant sprawling codebase feature packed software packages that define the modern software landscape. What's being missed is that any one user might only utilize 5% of the code base on any given day. Software is written to accommodate every need every user could have in one package. Then the users just use the small slice that accommodates their specific needs.
With all due respect, the fact that you made a few small programs to help with your tasks is wonderful but this last statement alone rather disqualifies your expertise to make an assessment on software engineering in general.
There's a great number of reasons why codebases get large. Complex problems inherently come with complexity and scale in both code and integrations. You can choose to move the complexity around but never fully get rid of it.
At a very conservative guess I'd say no more than 10% (and my actual guess would be <1%)
A recent example from a C# project I was working in. The project used builder classes that were constructed according to specified rules, but all of these builders were written by hand. I wanted to automatically generate these builders, and not using AI, just good old meta-programming.
Now I knew enough to know that I needed a C# source generator, but I had absolutely no experience with writing them. Could I have figured this out in an hour or two? Probably. Did I write a prompt in less than five minutes and get a source generator that worked correctly in the first shot? Also yes. I then spent some time cleaning up that code and understanding the API it uses to hook into everything and was done in half an hour and still learnt something from it.
You can make the argument that this source generator is in itself "boilerplate", because it doesn't contain any special sauce, but I still saved significant time in this instance.
I can't even fathom how frustrating such tools would be with poorly written confusing Clojure code using some niche dependency.
That being said, I can imagine a whole class of problems where this could succeed very well at and provide value. Then again, the type of problems that I feel these systems could get right 99% of the time are problems that a skilled developer could fix in minutes.
This project is not your typical Webdev project, so maybe that's an interesting case-study. It takes a C-API spec in JSON, loads and processes it in Python and generates a C-library that turns a UI marked up YAML/JSON into C-Api calls to render that UI. [1]
The result is pretty hacky code (by my design, can't/won't use FFI) that's 90% written by Gemini 2.5 Pro Pre/Exp but it mostly worked. It's around 7k lines of Python that generate a 30-40k loc C-library from a JSON LVGL-API-spec to render an LVGL UI from YAML/JSON markup.
I probably spent 2-3 weeks on this, I might have been able to do something similar in maybe 2x the time but this is about 20% of the mental overhead/exhaustion it would have taken me otherwise. Otoh, I would have had a much better understanding of the tradeoffs and maybe a slightly cleaner architecture if I would have to write it. But there's also a chance I would have gotten lost in some of the complexity and never finished (esp since it's a side-project that probably no-one else will ever see).
What worked well:
* It mostly works(!). Unlike previous attempts with Gemini 1.5 where I had to spend about as much or more time fixing than it'd have taken me to write the code. Even adding complicated features after the fact usually works pretty well with minor fixing on my end.
* Lowers mental "load" - you don't have to think so much about how to tackle features, refactors, ...
Other stuff:
* I really did not like Cursor or Windsurf - I half-use VSCode for embedded hobby projects but I don't want to then have another "thing" on top of that. Aider works, but it would probably require some more work to get used to the automatic features. I really need to get used to the tooling, not an insignificant time investment. It doesn't vibe with how I work, yet.
* You can generate a *significant* amount of code in a short time. It doesn't feel like it's "your" code though, it's like joining a startup - a mountain of code, someone else's architecture, their coding style, comment style, ... and,
* there's this "fog of code", where you can sorta bumble around the codebase but don't really 100% understand it. I still have mid/low confidence in the changes I make by hand, even 1 week after the codebase has largely stabilized. Again, it's like getting familiar with someone else's code.
* Code quality is ok but not great (and partially my fault). Probably depends on how you got to the current code - ie how clean was your "path". But since it is easier to "evolve" the whole project (I changed directions once or twice when I sort of hit a wall) it's also easier to end up with a messy-ish codebase. Maybe the way to go is to first explore, then codify all the requirements and start afresh from a clean slate instead of trying to evolve the code-base. But that's also not an insignificant amount of work and also mental load (because now you really need to understand the whole codebase or trust that an LLM can sufficiently distill it).
* I got much better results with very precise prompts. Maybe I'm using it wrong, ie I usually (think I) know what I want and just instruct the LLM instead of having an exploratory chat but the more explicit I am, the more closely the output is to what I'd like to see. I've tried to discuss proposed changes a few times to generate a spec to implement in another session but it takes time and was not super successful. Another thing to practice.
* A bit of a later realization, but modular code and short, self-contained modules are really important though this might depend on your workflow.
To summarize:
* It works.
* It lowers initial mental burden.
* But to get really good results, you still have to put a lot of effort into it.
* At least right now, it seems you will still eventually have to put in the mental effort at some point, normally it's "front-loaded" where you have to do the design and think about it hard, whereas the AI does all the initial work but it becomes harder to cope with the codebase once you reach a certain complexity. Eventually you will have to understand it though even if just to instruct the LLM to make the exact changes you want.
But I have done larger tasks (write device drivers) using gemini.
In short, the tools work. I've built things 10x faster than doing it from scratch. I also have a sense of what else I'll be able to build in a year. I also enjoy not having to add cycles to communicate with external contributors -- I think, then I do, even if there's a bit of wrestling. Wrangling with a coding agent feels a bit like "compile, test, fix, re-compile". Re-compiling generally got faster in subsequent generations of compiler releases.
My company is building internal business functions using AI right now. It works too. We're not putting that stuff in front of our customers yet, but I can see that it'll come. We may put agents into the product that let them build things for themselves.
I get the grumpiness & resistance, but I don't see how it's buying you anything. The puck isn't underfoot.
Lots missing here, but I had the same issues, it takes iteration and practice. I use claude code in terminal windows, and text expander to save explicit reminders that I have to inject super regularly because anthropic obscures access to system prompts.
For example, I have 3 to 8 paragraph long instructions I will place regularly about not assuming, checking deterministically etc. and for most things I have the agents write a report with a specific instruction set.
I pop the instructions into text expander so I just type - docs when saying go figure this out, and give me the path to the report when done.
They come back with a path, and I copy it and search vscode
It opens as an md and i use preview mode, its similar to a google doc.
And ill review it. always, things will be wrong, tons of assumptions, failures to check determistically, etc... but I see that in the doc and have it fix it. correct misunderstandings, update the doc until its perfect.
From there ill say add a plan in a table with status for each task based on this ( another text expander snippet with instructions )
And WHEN thats 100% right, Ill say implement and update as you go. The update as you go forces it to recognize and remember the scope of the task.
Greatest points of failure in the system is misalignment. Ethics teams got that right. It compounds FAST if allowed. you let them assume things, they state assumptions as facts, that becomes what other agents read and you get true chaos unchecked.
I started rebuilding claude code from scratch literally because they block us from accessing system prompts and I NEED these agents to stop lying to me about things that are not done or assumed, which highlights the true chaos possible when applied to system critical operations in governance or at scale.
I also built my own tool like codex for managing agent tasks and making this simpler, but getting them to use it without getting confused is still a gap.
Let me know if you have any other questions. I am performing the work of 20 Engineers as of today, rewrote 2 years of back end code that required a team of 2 engineers full time work in 4 weeks by myself with this system... so I am, I guess quite good at it.
I need to push my edges further into this latest tech, have not tried codex cli or the new tool yet.
its a pain but it works.
Even TDD it will hallucinate the mocks without management. and hallucinate the requirements. Each layer has to be checked atomically, but the text expander snippets done right can get it close to 75% right.
My main project faces 5000 users so I cant let the agents run freely, whereas with isolated projects in separate repos I can let them run more freely, then review in gitkraken before committing.
You can customize the system prompts, baseline propmts, and models used for every single mode and have as many or as few as you want.
IMO LLMs are actually pretty good at writing small scripts. First, it's much more common for a small script to be in the LLM's training data, and second, it's much easier to find and fix a bug. So the LLM actually does allow a non-programmer to write correct code with minimal effort (for some simple task), and then they are blown away thinking writing software is a solved problem. However, these kinds of people have no idea of the difference between a hundred line script where an error is easily found and isn't a big deal and a million line codebase where an error can be invisible and shut everything down.
Worst of all is when the two sides of tech-giants and non-programmers meet. These two sides may sound like opposites but they really aren't. In particular, there are plenty of non-programmers involved at the C-level and the HR levels of tech companies. These people are particularly vulnerable to being wowed by LLMs seemingly able to do complex tasks that in their minds are the same tasks their employees are doing. As a result, they stop hiring new people and tell their current people to "just use LLMs", leading to the current hiring crisis.
Indeed, even codex (and i've been using it prior to this release) is not remotely at the level of even a junior engineer outside of a set of tasks.
ChatGPT is good for asking questions about languages, SDKs, and APIs, or generating boilerplate, but it's useless if you want to give an AI a ticket and for it to raise PRs for you.
This is where you need agentic solutions like Codex which will be far more useful because they will actually have access to your codebase and a dev environment where they can test and debug changes.
They still do really dumb things, but a lot of this can be avoided if you prompt well and give it the right types of problems to solve.
In my experience at the moment there's a sweet spot with these agentic coding platforms which makes them useful for semi-complicated tasks – assuming you prompt well they can generate 90% of the code you need, then you just need to spend the extra 10% fixing it up before it's ready for prod.
Tasks too simple (a few lines) it's a waste of time. You spend longer prompting and going back and forth with the agent than it would take to just make the change yourself.
Then obviously very complicated tasks, especially tasks that require some thought around architecture and performance, coding agents really struggle with. Less because they can't do it, but because for certain problems simply meeting ACs is far less important than how the ACs are being met. Ideally here you want to get the architecture right first, then once that's in place you can break down the remaining work for the AI to pick up.
If you haven't set things up properly (important info lives only in people’s heads / meetings, tasks dont have clear acceptance criteria, ...) then you aren't ready for Junior Developers yet. You need to wait until your Coding Agents are at Senior level.
I imagine many engineers are like myself in that they got into programming because they liked tinkering and hacking and implementation details, all of which are likely to be abstracted over in this new era of prompting.
This sparked a thought in how a large part of the job is often the work needed to demonstrate impact. I think this aspect is often overlooked by some of the good engineers not yet taking advantage of the AI tooling. LLM loops may not yet be good enough to produce shippable code by themselves, but they sure are capable to help reduce the overhead of these up and out communicative tasks.
> without sacrificing quality
Right..
> it's your responsibility to use that tool
Again, it's actually not. It's my responsibility to do my job, not to make my boss' - or his boss' - car nicer. I know that's what we all know will create "job security" but let's not conflate these things. My job is to do my end of the bargain. My boss' job is paying me for doing that. If he deems it necessary to force me to use AI bullshit, I will of course, but it is definitely not my responsibility to do so autonomously.
I guess that's LLMs ruled out then
people who want to make software that enables people to accomplish [task] will get the software they need quicker.
We had to tinker piece by piece to build a miniature castle. Over many hours.
Now I can tinker concept by concept, and build much larger castles, much faster. Like waving a wand, seeing my thoughts come to fruition in near real time.
No vanity lost in my opinion. Possibly more to be gained.
If you derive enjoyment from actually assembling the castle, you lose out on that by using the wand that makes it happen instantly. Sure wand's castles may be larger, but you don't put a Lego castle together for the finished product.
See that never was the purpose.. going bigger and faster, towards what exactly? Chaos? By the way we never managed to fully tackle manual software development by trained professionals and we now expect Shangri-La by throwing everything and the kitchen sink into giant inscrutable matrices. This time by amateurs as well. I'm sure this will all turn out very well and very, very productive.
1. When I work on side projects and use AI, sometimes I wonder "what's the point if I am just copy / pasting code? I am not learning anything" but what I have come to realize is building apps with AI assistance is the skill that I am learning, rather than writing code per se as it was a few years ago.
2. I work in high scale distributed computing, so I am still presented with ample opportunities to get very low level, which I love. I am not sure how much I care about writing code per se anymore. Working with AI still is tinkering, it has not changed that much for me. It is quite different, but the underlying fun parts are still present.
For example, in the last month or so, I added a job queue plugin. The ability to run multiple tasks that they demoed today is quite similar. The issue I ran into with users is that without Enterprise plans, complex tasks run into rate limits when trying to run concurrently.
So I am adding an ability to have multiple queues, with each possibly using different models and/or providers, to get around rate limits.
By the way, my system has features that are somewhat similar not only to this tool they are showing but also things like Manus. It is quite rough around the edges though because I am doing 100% of it myself.
But it is MIT Licensed and it would be great if any developer on the planet wanted to contribute anything.
On the other hand, if your job was writing code at certain companies whose profits were based on shoving ads in front of people then I would agree that no one will care if it is written by a machine or not. The days of those jobs making >$200k a year are numbered.
Despite software being in everything and harm to the public due to bad software has materialized every developer seems vehemently against professionalizing. Do you want a surgeon that went to surgeon bootcamp because "you dont need all those years in medical school to learn how to remove an appendix"? Do you even want an accountant who went to accountant bootcamp to do your taxes?
I also do not think this is limited to software engineering. Medical doctors and accountants have faced the squeeze in recent years too. There are tons of (bad) DO med schools opening up across the country that will be flooding the field before long, nurse practitioners and physicians assistants get to do more and more work that only doctors got to do, and more and more accounting is being offshored. The question is when things get so bad that even the powerful decide to actually do something about it.
There's so many of these "vibe coding" tools and there has to be real engineering rigor at some point. I saw them demo "find the bug" but the bugs they found were pretty superficial and thats something we've seen in our internal benchmark from both Devin and Cursor. A lot of noise and false positives or superficial fixes.
sigh
I just enabled on Settings > Connectors > Github
hoping that makes it work
... still doesnt work, is it geo-restricted maybe? idk
I know right
also no human to contact on support... tempted to cancel the sub lol i'll give them 24h
≠ available now to all pro users
Every -big- release they gatekeep something to pro I pay for it like every 3 months, then cancel after the high
when will i learn
Short version: Jevons's paradox means that the more coding you automate away for developers (like with compilers in the past), the more in-demand those developers are. (Until AGI when all bets are off, of course.)