IBM's new SWE agents for developers
research.ibm.com
research.ibm.com
The video shows a private fork of a pubic repository. The bug is real, but it was resolved in February 2023 and doesn’t seem like the solution was automated [1]
The bug has a stack trace attached with a big arrow pointing to line 223 of a backend_compat.py file. A quick grasp on this stack trace and you already know what happened and why, and how to fix this, but…
not for the agent. It seems to analyze the repository in multiple steps and tries to locate the class. Why did they even release this video?
so, they organize hackathons where devs build a hypothetical agentic framework nobody will dare use. So, mgmt can claim, look here what i have done to be agentic.
you should ask: would you dogfood your agent, and the answer is no way. these are meant purely for marketing purposes, as they dont meet an end user need.
just goes to show, it is all a big song-and-dance. much ado about nothing.
The term "agent" implies you can give the AI full access to your repos and fire the software engineers you're grudgingly paying six figures to.
The second is much more valuable to executives not wanting to pay the software people that demand higher salaries than virtually everyone else in the organization.
i am saying, the thing is snake-oil - a solution looking for a problem.
Long term value, I agree.
Fundraising, hard disagree.
This way it would at least look like it may work
we are at a phase where the early adopters have seen the writing on the wall.. ie that llms are useful for a limited set of usecases. but there are lots of late adopters who are still awestruck and not disillusioned yet.
I made a few minor edits, but I think we all know this is coming. This calls itself "for developers" for now, but really also it's "instead of developers", and at some point the mask will come off.
Let the AI write all the code and programmers will do the fixes.
> LLMs are a tool, nothing more, they don't magically imbue the user with competency.
Not a good take though, IMO. They're literally a tool that can teach you how to use them, or anything else.
> Not a good take though, IMO. They're literally a tool that can teach you how to use them, or anything else.
I disagree. In their current incarnation, LLMs require a human subject matter expert to determine if the output is valid. In the project manager team lead example, the LLM won't tell you if the database is sized correctly, or if you even need a database.
This is 100% the play.
Right now you can hire 5 devs in India to do the job of 1 competent US dev and save 30-40% on total cost.
Add in AI and it will only take 3 devs in India to do the same work, and can now save 50-60% on total cost.
I have very little fear for my own job no matter how good models get. What happens is that software gets cheaper and more of it is bought. It’s what happened in every industry with automation.
Those who can’t operate a machine though (in this case an AI) should maybe worry. But chances are their jobs weren’t very secure to begin with.
Although I also wonder about the development of new languages that may be optimized for transformers, as it seems clumsy and wasteful to have transformers juggle all the tokens needed to make code readable by humans. That would be really interesting to have a model that outputs code that functions incredibly but is indecipherable by humans.
I don't think junior devs are going to benefit; if anything, the whole role of 'junior' has been made obsolete. The rote / repetitive work a junior would traditionally do, can now be delegated wholesale to a LLM.
I figure, productivity is going to be increased a lot. We'll need less developers as a result. The duties associated with developers are going to morph and become more solutions / architecture orientated.
I am from LogicStar.ai and we are trying to get rid of the BS and show true value in real-world applications with deep program analysis, reproduction, verification etc and like everyone else sourcing fix-suggestions from LLMs.
Follow our page on https://www.linkedin.com/company/logicstar-ai/ as I plan to post updates and frank comments here on what other agents are or are NOT. Your input will drive in what direction we should focus the technology and use cases.
They're not even in the top half of the leaderboard. Almost half the score of the first place agent.
What a weird sentence. Mx. Puri does not argue anything, this is just an unfounded claim. So far it just looks like snake oil that is to be sold to other companies.
This would actually be a good business strategy: Sell software that diminishes productivity to your competition and watch them disintegrate.
All i got instead are lame tools for developers.
Or is this the next iteration of static analysis?
These and other models IBM is working on can do basic tasks that anyone else could. But it will all fall apart the moment you add complexity to it.
It's hilarious to see how IBM struggles to stay relevant, what did that lead to? A bot that summarizes stack trace. Why is this even on the front page of HN?