Devin: AI Software Engineer
cognition-labs.com
cognition-labs.com
Even so, I realize the demos are still broad in scope and the results are incredible. Imagine seeing this even 2 years ago. It would seem like magic; you wouldn't be able to believe it. Today, this was inevitable and entirely believable. There will be even better versions of this soon.
In my mind, I've concluded that I have less than 3 years to find an off ramp.
I have a multi-part plan. Immediately, i'm working to get closer to the business. To be closer to the position of defining requirements, not implementing requirements.
Secondary, i'm experimenting with ideas I hope can become a business.
as a final fallback, I have a hobby woodshop in my basement, and I love making furniture.
Programmers will be fine. The AI plagiarism engines are severely limited, and will be for the forseeable future. Maybe someday i'll be equivalent to the ren-faire blacksmith, but I'm gonna do this until I die.
I'm looking for an offramp as well. I truly love software so it's a hard reality to contend with at times. Regardless, I'm not a software engineer at my core. I'm a problem solver, and I love creating things. This is transferable. I'm not sure where to yet, but I'm certainly capable of making a move. I hope I have more than 3 years, though.
As someone else asked, I'm curious where you're headed or thinking of heading so far.
I think the immediate future is bright. Some will try to cram this technology into enterprise. It will do well. Those jobs will die. Others will leverage it alongside the more creative engineering tasks—they will thrive.
Eventually, what we call software will change from what it is today into something much more accessible to these types of tools. The plateau we've landed on is just a compromise between the technology, economies, and culture of its time.
As this type of tech pervades our everyday lives, much of the widespread need for specialized software will evaporate. The remaining work will be in the corners or the edges of what's possible—highly technical or vertically integrated work (particularly, hardware-integrated stuff), as well as platform engineering to sustain the new paradigm.
That's a very good point. There's that tongue-in-check assertion that Java as a language isn't a human-facing but rather a tool-facing programming language. And now I'm wondering, what would a high-level language that is actually designed from the ground up for manipulation by AIs rather than humans look like. And in contrast, I think that the human-facing interface to it might end up much more visual and graph-oriented, possibly similar to Unreal Engine's blueprints.
I expect that we're moving into a phase of AIs talking to AIs, and initially it'll be wasteful (because it'll be mostly English), but eventually, they'll derive their own language and seamlessly upgrade protocols when they determine they're talking to an AI. No clue how that will come about or what that language will look like, but honestly, it's kind of exciting.
Really interesting to think about how they might handle context, as well. Even though we have much bigger context windows (and they'll only get larger), context management is still a resource-management issue, which we'll probably continue to refine, as well. Imagine different strategies for managing both what is brought into the context of each request, as well as what form it could take (level of detail, additional references or commentary on it, etc). Things could get really unreadable even in English, and still be very interpretable for an LLM.
W.r.t. the graph-oriented interfaces, are you thinking something like Node-RED [1]? I'm seeing more and more people mention having LLMs produce non-text or structured outputs, like JSON, UI, and other things. Easy to imagine an LLM that wires together various open-source platforms, on-demand. Something like Node-RED for pipelines/functions, some UI tools for visualization/interactivity, other platforms for messaging, etc...
This change is roughly analogous to the shift from punch cards to compilers. It's just a more "natural language" way to do programming. A lot of the drudgery associated with coding will go away and competent programmers will shift to higher level design work.
Even in a future where AI is better than human software engineers at every single programming task (which I don't believe will be the case any time soon), there is still comparative advantage. AI will not have the _capacity_ to do every single programming task and there's going to still be lots of work for people to do.
THis is exactly what I want.
I dont want to describe some enterprise architure with DR and blah.
No - I want the little utils that I just want to appear like little magic fairy dust...
Little apps. This is how a new eco system grows.
As a tangent, I've been thinking about this in regards to the edtech space. For now, the focus is on GenAI tools for quickly building courses, but if we have AI with the combined skills of a Subject Matter Expert, Instructional Designer and Tutor, why would we need to build courses at all, if instead it could just teach anyone anything on the fly, based on their specific background and interests, and adapting immediately to their reactions?
We've already been talking a lot about bubbles / echo chambers over the last decade, and I imagine that this will get a lot worse/better over the coming decade, depending on your perspective of the value of shared experiences.
"Generative AI will mean that Pixar movies will be made by AI!" Likelier it means that generative AI will create an entirely new medium beyond movies.
Now everyone seems to be a "doomer" like me. I was too stupid to see that people (especially online) were lashing out at me as a coping mechanism. The worst part is that even though I had my "we're fucked" realization was earlier than many others, I didn't really do anything productive, I didn't make the right moves to pivot, I had no plan. Now I have a rough ~3-year off ramp plan, just like everyone else.
I suspect that there are many, many recent grads in my position, who also thought they were taking crazy pills when everyone around them dismissed their concerns
They don't know how the industry works or what it needs, few of them them know how the technology works or what it's realistic near-term prospects are let alone the long-term ones, and none of them have lived long enough to understand the pace of a technology moving from discovery through to maturity and commercialization.
If you look to your experienced seniors instead of your peers, you'll discover two things:
1. They don't respect most contemporary juniors because of the industry boom and are happy to see most of you scare yourselves off
2. They're narrowing in on a familiar, respectful appreciation of recent content generators and their potential for assisting some basic use cases (highly valuable technology!) and are increasingly of the opinion that both the hype and fear of Q1/2023 was largely market manipulation by VC's trying to squeeze another boom out as money was tightening
But something changed in the past 4 months or so. I used to see the two things you described, but now I see seniors repeating the same thing juniors (whom they ridiculed) used to say. Look at this very thread, for example. People are starting to realize that even if these AI startups are making ridiculous promises, the tech might be way more than just hype/smoke and mirrors.
I've read quite a few comments in HN, and right here in this thread, that say something along the lines of "I'll be fine... It's the juniors I'm worried about".
"Surely if we generate code, it means we can replace developers right?"
Well no, it's not nearly enough. Good code generation with a UI is already a thing since the 90s, it's replacing developers as much as Frontpage and WordPress do.
The average developer writes 100 lines per day anyways so the speed of writing those lines doesn't matter.
Software is an interpersonal job at core and the most precise definition of a business.
The day software engineers are replaced by a machine, there's no job for the CEO itself honestly and that's also why this job is well paid. I'm sure it will be all over LinkedIn though, despite this paradox.
Those new tools will be there in the background but cannot replace a software engineer.
Somewhat related: is anyone else not really impressed by Devin fixing the errors that are very preventable with a stricter language like Rust? The demo shows Devin coding in both Python in Rust, but I consider the latter being way less energy intensive in terms of maintenance. Then again, exhaustive pattern matching and strict typing won't get you lots of VC dollars these days.
Just yesterday I tried to feed it a simple HTML page to extract a selector, I tried it with GPT-4-turbo, I tried it with Claude, I tried it with Groq, I tried it with a local LLama2 model with 128k context window. None of them worked. This is a task that while annoying, I do in about 10 seconds.
Sure, I'm open to the possibility that in the next 2 - 3 days up to a couple of years, I'll no longer do manual coding. But honestly. After so much hype, I'm starting to grow a bit irritated with the hype.
Just give me a product that works as advertised and I'll throw money your way because I have a lot more ideas than I have code throughoutput!
Honestly, LLMs can't even get language right. They produce generic, amateurish copy that reads like it's written by committee. GPT can't perform to the level of a middle market copywriter or content marketer. I am convinced that people who think LLMs can write have simply not understood what professional writers do.
For me the "plateau of productivity" after the disillusionment has been using LLMs a bit like search engines. Quick standalone summaries, snippets or thoughts. A nice day-to-day productivity boost, but nothing that's going to allow me to work less hard.
GPT's rigid "robot butler" style is not "just how LLMs write". OpenAI deliberately tuned it to sound that way. Even much weaker models that aren't tuned to write in a particular way can easily pass for human writing.
It's funny to see, because developers are so sensitive about being treated like code monkeys by their non-technical colleagues. But these same devs turn around to treat other professionals as word monkeys, or pixel monkeys, or whatever else. Not realizing that they are only seeing the tip of the iceberg of someone else's profession.
Professional writers don't take prompts and shit out words. They work closely with their clients to understand the important outcomes, then work strategically towards them. The dead giveaway of LLM writing isn't the style. It's the lack of coherent intent behind the words, and low information density of the text. A professional writer works to communicate a lot with very little. LLMs work in the opposite way: you give it a prompt, then it blows it out into verbiage.
Sit down for coffee with a professional copywriter (not the SEO content marketing spammers), and see what they have to say about LLMs.
Personally, I group all these things under 'style'. Perhaps, i should have used, 'presentation' instead. You've latched on that specific word and gone off. Point is that the post-training of these models, especially GPT from Open ai is doing a lot to how the writing (the default at least) presents long strings of text. Like how GPT-4 is almost compelled to end bouts of fiction prematurely in sunshine and rainbows. That technically isn't style but is part of what i was talking about.
>A professional writer works to communicate a lot with very little. LLMs work in the opposite way: you give it a prompt, then it blows it out into verbiage.
There's no reason you have to work this way with an LLM.
No, I haven't. I'm not talking about style, but something deeper. What I'm talking about is something you don't even seem to realize exists in professional writing - which is why you keep thinking I'm misunderstanding you when I am not.
I've worked with professional writers, and nothing in the LLM space even comes close to them. It's not a matter of low quality vs high quality, or benchmarking, or style. It's simply an apples and oranges comparison.
The economics of LLMs for shortform copy will never make sense, because producing the words is the cheapest part of that process. They might become the best way for writers themselves to produce longform copy on the execution side, but they can't replace the writer's ability to work with the client to figure out exactly what they are trying to write, and why, and what a good result even looks like. And no, this isn't a prompting issue, or a UI issue, or a context window length issue, or anything like that.
Elsewhere in this thread someone mentioned how invaluable LLMs are for producing internal business copy. I could easily see these amateur writing tasks being replaced by LLMs. But the implication there isn't that LLMs are any good at writing, but that these tasks don't require good writing to begin with.
I've read hundreds of books, fiction and otherwise. This isn't a brag, it's just to say, believe me, I know what professional writing looks like and I know where LLMs currently stand because I've used them a lot. I know the quality you can squeeze out if you're willing to let go of any presumptions.
You'll notice that not once did I say current LLMs could wholesale replace professional writers anymore than they can currently replace professional software devs. I just disagree on the "not a good writer" bit.
If it's the opinion of professional writers you're looking for then you can find some who disagree too.
Rie Kudan won an award on a novel she used GPT to verbatim ghostwrite (no edits essentially) 5% of. Her words, not mine. Who knows how much more of the novel is edited GPT.
That a professional human novelist was able to leverage GPT for their book isn't disproving the grandparent's post. They knew what good looks like, and if it wasn't good they wouldn't have kept it in the book.
When part of the OPs point seems to be that LLMs can't write good stuff then that's proof enough.
If you're talking about replacing professionals wholesale then I never made that argument.
LLMs can't write good stuff. Human writers can write good stuff. When a good writer uses an LLM in their writing process, that writer can certainly produce good writing.
When an AI hypebro who is otherwise a bad writer uses an LLM in their writing process, they still produce bad writing.
Waiting for the Author who has used a Markov Chain to ghost write.
>LLMs can't write good stuff. Human writers can write good stuff. When a good writer uses an LLM in their writing process, that writer can certainly produce good writing.
Give it a rest. The author was quite clear she copy pasted sections of writing in.
Then the humans can make a heroic return, write surgical prose like Hemingway to slice through the AI drivel, and keep collecting their paychecks.
Bonus points if you can translate this analogy to software development...
I've had the same experience as well. I heard tons of people clamoring about the ability for LLM's to write SEO copy for you and how you can churn out web content so much faster now. I tried using it to churn out some very specific blog posts for an aborist client of mine.
The results were really bad. I had to re-write and clarify a lot of what it spit out. The grammar was not very good and it was really hard to read with very poorly structured sentences that would end aburptly and other glaring issues.
I did this right after a guy I play hockey with said he uses it all the time to write emails for him and pays the monthly subscription in order to have it write all kinds of things for me every day. After my trial, I was really wondering how obvious it was that he was doing that and how his clients thought about him knowing how poorly the stuff these LLM's were putting out.
>After my trial, I was really wondering how obvious it was that he was doing that and how his clients thought about him knowing how poorly the stuff these LLM's were putting out.
I feel the same way about this stuff as when devs say they push out LLM code with no refactoring or review. Ah, good luck!
If you were only using GPT 3.5 (free ChatGPT) then your opinion is irrelevant.
With GPT-4 you could directly ask it: "rewrite your previous response so that it sounds less generic, less amateurish, and not written by a committee". I'm not even joking. Just provide enough information and tell it what to do. If you don't like the output then tell it what needs to be improved. It's not a mind reader.
Also GPT-4 is a year old now. Claude 3 is already superior and GPT-5 will be next level.
The model names, version numbers or who makes them are irrelevant.
If you use GPT-4 day-to-day, you've probably encountered this sense of a capability wall before. The point where additional prompting, tweaking, re-prompting simply doesn't seem to be yielding better results on the task, or it feels like the issue is just being shifted around. Over time, you develop a bit of a mental map of where the strengths and weaknesses are, and factor that into your workflows. That's what writing with LLMs feels like, compared to working with a professional writer.
Most writers have already realized that LLMs can't write in any meaningful way.
I know a professional writer who is amazed by what LLMs are capable of already and, given the rate of progress, speculates they will take over many writing jobs eventually.
> If you use GPT-4 day-to-day, you've probably encountered this sense of a capability wall before.
Of course there is a wall with the current models. But almost every time I hit a wall, I have found a way to break past that limit. Interacting with the LLM as I would interact with a person. LLM's perform best with chain of thought reasoning. List out any issues you identified in the original output, ask the LLM to review these issues and list out any other issues that it can identify based on the original requirements, then rewrite it all. And do that several times until it's good enough.
At work I have found GPT-4 to exceed the linguistic capabilities of my colleagues when it comes to summarizing complicated boring business text.
Or just write the damn thing yourself.
[1]https://github.com/xenodium/chatgpt-shell/blob/bf2d12ed2ed60...
Ask it to "rewrite that in the tone of an English professor" or "rewrite that in the style of a redneck rapper" or "make that sound less like generic ad copy". Get into an argument back and forth with the LLM and tell it the previous response is crap because of XYZ.
And it only took one of the most computationally expensive processes ever devised by man.
In some circles, that's actually a wonderful achievement. :p
In the realm of coding here are a few things its really good at:
- Translating code, generating cross language clients. I'll feed it a golang single file API backend and tell it to generate the typescript client for that. You can add hints like e.g "use fetch", "allow each request method to have a header override", "keep it typesafe, use zod", etc
- Basic validation testing. It's pretty good at generating scaffold tests that do basic validation (Opus is good at writing trickier tests) as your building.
- Small module completion. I write an interface of a class/struct with it's methods and some comments and tell it to fill in. A recent one I did looked something like (abbreviated):
type CacheDir struct { dir string, maxObjectLifetime: Duration, fileLocks sync.Map }
type (cd *CacheDir) Get(...)
type (cd *CacheDir Set(...)
type (cd *CacheDir) startCleanLoop()
Opus does a really good job generating the code and basic validation tests for this.
One general tip: you have to be comfortable spending 5 minutes crafting a detailed query assuming the task takes longer than that. Which can be weird at first if you take yourself seriously as a human.
Note that I hadn't been able to do much of this with GPT-4 Turbo with with Claude Opus it really feels capable.
So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along.
Gemini Advanced is also a great model, but for other reasons. That thing really knows a boat load of low level optimization tricks.
It's that hype-churn marketing that is a motivating factor for the groups to innovate, much like Formula 1. It might be distasteful, but that doesn't mean it isn't working.
So what is the SOTA, in your opinion?
I'm talking about 4-turbo, 4-turbo preview and self hosted LLama2. What in God's name is not SOTA about this?
Good fucking Christ, Opus has been out for like 8 days, I've had holidays longer than that!
Can any convert a native iPhone app to an Android one?
Are they considerably better than existing non-AI tools + manual coding for this? In VSCode and Visual Studio, when working with an interface in C# for example, I can click two context menus to have it generate an implementation with constructors, getters, & setters included, leaving only the business logic code to write manually. You've mention you have to describe to the AI in comments, and then I assume spend time on a step to verify the AI has correctly interpreted your request & implemented.
I can definitely see the advantage for LLMs when writing unit tests on existing code, but short of very limited situations, I'm really finding it difficult to find the 55% efficiency improvements claimed by the likes of GitHub's AI Copilot.
Totally agree on the 55% figure being hogwash.
But the “you won’t need to code” reminds me “you won’t need to learn to drive”.
It’s the messy interface with the real world in both cases that basically requires AGI.
If AGI is just a decade off then, yep, I won’t need to code. But a decade is a long time and, more importantly, we’re probably more than a decade away.
And even if it is “just round the corner”, worrying about not needing to code would be worrying about deckchairs on the titanic. AGI will probably mean the end of capitalism as we know it, so all bets are off at that point.
It’s wise to hedge a little but also realise that to date AI is just a coding productivity boost. The size of the boost depends on how trivial the code is. Most of the code I write isn’t trivial and AI is fairly useless at that, certainly it’s faster and more accurate to write it myself. You can get a 50% boost if you’re writing boiler plate all day, but then you have to wonder why you’re doing that in the first place.
The point at which someone like myself is allowed to become aware that a company has developed that level of AI is well beyond the point of no return.
Like, by now you'd think it would be inevitable that we wouldn't be writing software in a text-editor or IDE. Everything else we do on a computer is more graphical rather than textual, with the exception of software development. Why is that?
Part of the reason why I'm kind of bearish on AI is because it seems like we could have replaced written code with GUI diagrams as far back as the 80s, or at the very least in the early 2000s, and it seems like something that should have obviously caught on given that would probably be much easier for the average person. Again though, curiously, we're still using text editors. Perhaps despite the popularization of AI no-code builders we'll still see that the old model of hiring someone good at writing code in a text-editor remains largely unchanged.
Makes me wonder if there's just something about the process that we overlook, and if this same something could frustrate attempts at automating the process of writing code using AIs as much as it frustrated our attempts at capturing code using graphical symbols.
I don't think most people are building landing pages anymore by handwriting code anymore. Same with blogs (eg. Wordpress). There are MVPs of successful businesses that've been built by Bubble.io. Internal dashboards and such can definitely be built without code such as via Retool or Looker or whatever.
WYSIWYG obviously makes sense for frontend, but less so for backend. For backend code I don't really see how some visual drag and drop editor could make for a better interface than code. And even if it could, the advantage of code is that it's fully customizable (whereas with a GUI you're limited by the GUI), and text itself as a medium is uniform and portable (eg. easy to copy and paste anywhere).
Not to say that we can't create better interfaces than text, but I do think some sort of augmentation on top of a code editor is probably a more realistic short-term evolution, similar to VSCode plugins.
Write me the molecular simulation boilerplate because these crappy tools all have their own esoteric DSLs, then I tweak the parameters to my use case, avoiding the busywork -
e.g. "Write me a simulation for methane burning in air"
Gives me a boilerplate, I modify the initial conditions (concentrations, temperatures, etc) and then deploy. Have the LLM do the busy-work, so I dont have to spend ages reading docs or finding examples just to get started.
Now deploy to a stable environment. Thats what I'm trying to help with by building https://atomictessellator.com
What is worse is that seems to be leading to a self-amplifying feedback loop, where people not up to speed enough with the models try to use them, fail and give up making them fall even further behind.
Also, current crop of LLMs are not there yet for large/largish projects. GPT4 is too slow and expensive, while Groq is superfast but open source models are not quite there yet. Claude is somewhere in the middle. I expect somewhere in the next 12 months there's going to be a tipping point where they will be capable, fast, and reliable enough to be in wide use for coding in this style[1].
[0] I have an AI horse in the game with http://pythagora.ai, so yeah I'm biased [1] It already works well for snippet-level cases (eg GitHub copilot or Cursor.sh) where you still have creative control as a human. It's exponentially harder to have the AI be (mostly) in control.
But roughly on par with what could be expected of today's junior software developer (unaided by AI)? Definitely.
- replacing StackOverflow and library documentation
- library search
- converting between formats and languages
- explaining existing code/queries
- deobfuscating code
- explaining concepts (kinda hit or miss)
- helping you get unstuck when debugging or looking for solution (‘give me possible reasons for …’)
I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant.
Library documentation? Yeah, it doesn't really save time when GPT makes up functions and libraries, making me check the docs anyways...
I was initially hopeful but I find it gets in my way for anything not trivial.
That behavior is now vanishingly-rare, at least in GPT4.
Did you use an AI assistant while making that list?
I find it horrible at replacing library documentation
> I feel like many of this things require asking the right questions, which assumes certain level of experience. But once you reach this level, it’s an extremely valuable assistant.
I've been using LLM products since incipience. I use them in my daily work life. It's a bit tiring hearing this 'right questions', 'level of experience' and 'reach this level'. Can you share anything concrete that you achieved with ChatGPT that would blow my mind?
I keep hearing this 'you need to ask the right kind of questions bro' from people that never build a single product in their life, and it makes me question my ability to interact with LLM but I never see anything concrete.
GPT reconstruction:
The dream reaches its peak when you meet the "final boss" of the game: an entity that embodies the ultimate barrier to communication. To overcome this obstacle, you must synthesize everything you've learned about "it" in the dream and present a coherent vision that is true to yourself. As you articulate your final understanding of "it", the maze dissolves around you, leaving you in front of a giant mirror. In this mirror, you see not just your reflection but also all the characters, passions, and ideas you encountered in the dream. You realize that "it" is actually a reflection of yourself and your ability to understand and share your inner world. The dream ends with the title revealed, "Game to Learn to Communicate about It and Its Player", meaning the whole process was a metaphor for learning to know and communicate your own "it" - your personality, thoughts, and emotions - with others, and that you are both the creator and the discoverer of your own communication game.
My note:
The continuation of the dream corresponds to an abrupt change of scene. I find myself in my bed, in the dim light of my room, facing a mysterious silhouette. As I repeatedly inquire about its identity, I stretch my hands towards its face to feel its features as I cannot clearly see them. Then, a struggle begins, during which I panic, giving the dream a nightmarish turn. Noticing that the dark figure mirrors my movements, I realize it's myself. Suddenly under my duvet and as I struggle to get out, I feel jaws and teeth against the sheets. I call out for my mother, whom I seem to hear downstairs, and that's when my vision fades, and I see the dream's source code displayed behind. It consists of ChatGPT prompts shared on the lime green background of an image-board. At the bottom, I then see the dream's title: "Game to learn how to communicate about It and its player."
In less time that it would take me to figure out how dictionary comprehension work in python. But hey, I guess you already know about these use cases.
So you can increase the quality of the response by writing your question in the same kind of style you would see in great documentation. Include lot's of details, be very specific, ask about one thing.
1) copilot is a terrific auto complete, and writes tremendous amounts of repetitive boilerplate
2) copilot can help me kickstart writing some complex functions starting from a comment where I tell it what is the input and expected output. Is the implementation always perfect or bug free? No. But in general I just need to review and check rather than come up with the instruction entirely.
3) copilot chat helps me a lot in those situations where I would've googled to find how to do this or that and spent a lot of time with irrelevant or outdated search results
4) I have found use cases for LLMs in production. I had lots of unformatted plain text that I wanted to transform in markdown. All I needed to do is to provide few examples and it did everything on its own. No need to implement complex parsers, but make a query to OpenAI with the prompt and context. Few euros per month in OpenAI credits is still insanely cheaper than paying tons of money in writing and maintaining software by humans for that use case.
5) It helps me tremendously when trying to learn new programming languages or remembering some APIs. Writing CSS selectors is actually a very good example. But I don't feed it an entire HTML as you do, I literally tell him "how do I target the odd numbered list elements that are descendants of .foo-bar for this specific media query". Not sure why would you need to feed it an entire HTML.
6) LLMs have been extremely useful to generate images and icons for an entire frontend application I wrote
7) I instruct him to write and think about test cases about my code. And it does and writes the code and tests. Often thinks about test cases I would've never thought of and catches nice bugs.
I really don't buy nor think it can write too much on its own.
The promise of it writing anything but simple boilerplate, I find it ridiculous because there's way too much nuance in our products, business, devices, systems that you need to follow and work on.
But as a helper? It's terrific.
I'm 100% sure that people not using these tools are effectively limiting themselves and their productivity.
It's like arguing you're better off writing code without a type checker or without intellisense.
Sure you can do it, but you're gonna be less effective.
1) Autocomplete is more often than not what I want or pretty darn close.
2) Sometimes I need a discrete function that I am not sure how I want to write. I use a prompt with 3.5/4 inside of my IDE to ask it to write that function.
It is definitely not writing complete programs any time soon but I can see where it's heading in the near term. Couple it with something like RAG to answer questions on library/api implementations. Maybe give it a stronger opinion about what good Python code looks like.
For the naysayers I don't know how you use it but it is certainly useful enough for me to pay for.
I agree. I have it active on VSCode and enjoy it. It has introduced subtle bugs but the souped up autocomplete is nice.
> 2) copilot can help me kickstart writing some complex functions starting from a comment where I tell it what is the input and expected output. Is the implementation always perfect or bug free? No. But in general I just need to review and check rather than come up with the instruction entirely.
I don't find it very useful for anything non trivial. If anything I found it more useful for generating milestones and tasks for a product, than even making a moderately complex input -> output without me having to check it in a way that annoys me.
> 3) copilot chat helps me a lot in those situations where I would've googled to find how to do this or that and spent a lot of time with irrelevant or outdated search results
I find I don't use copilot chat, almost at all. Nowadays I prefer to go to Gemini and throw in my question.
> 4) I have found use cases for LLMs in production. I had lots of unformatted plain text that I wanted to transform in markdown. All I needed to do is to provide few examples and it did everything on its own. No need to implement complex parsers, but make a query to OpenAI with the prompt and context. Few euros per month in OpenAI credits is still insanely cheaper than paying tons of money in writing and maintaining software by humans for that use case.
This is mostly what I'm using it for in this current project. It does it job nicely but it's very far away from replacing myself as a programmer. It's more like a `fn:magic(text) -> nicer text`. This is a good use case. But it's a tool, not a replacement.
> 5) It helps me tremendously when trying to learn new programming languages or remembering some APIs. Writing CSS selectors is actually a very good example. But I don't feed it an entire HTML as you do, I literally tell him "how do I target the odd numbered list elements that are descendants of .foo-bar for this specific media query". Not sure why would you need to feed it an entire HTML.
Because I get random websites with complex markup, and more often than not every page has its unique structure. I can't just say give me `.foo-bar` because `.foo-bar` might not exist. Which is where the manual process comes in. Currently, I'm using hand crafted queries that get fed into GPT / Claude / LLama, but the actual query is what I wanted it to do.
> 6) LLMs have been extremely useful to generate images and icons for an entire frontend application I wrote
I'm very curious how this behaves in different resolutions. There's a reason vector graphics are a thing. I've used it for this purpose before but it doesn't compare to vectorial formats.
> 7) I instruct him to write and think about test cases about my code. And it does and writes the code and tests. Often thinks about test cases I would've never thought of and catches nice bugs.
What is the context size of your code? It works for trivial snippets but as soon as the system is a bit more complex, I find that it becomes irellevant fairly fast.
> The promise of it writing anything but simple boilerplate, I find it ridiculous because there's way too much nuance in our products, business, devices, systems that you need to follow and work on.
> But as a helper? It's terrific.
> I'm 100% sure that people not using these tools are effectively limiting themselves and their productivity.
Totally agree. But I'm not complaining about its usefulness. I'm a paying user of LLM systems. I use them almost every day. They're part of my products. But this particular hype about it replacing ... me. I don't buy. Yet. It could come tomorrow and I'd be happier for it.
The sweet spot for me is using a plug-in within the IDE that utilizes an API key to the model API. That coupled with the ability to customize the system prompt has been amazing for me. I truly dislike all of the web interfaces, just allow me to pick from some predefined workflows with a system prompt that I create and let me type. Within the IDE I generally have it setup so that it stays concise and returns code when asked with minimal to no explanation. Blazing efficient.
Especially #5. I'm certain that I've been at least 10x more productive in learning new tools since chatgpt hit the scene. And then knowing it helps so much with that has had even more leverage in opening up possibilities for thing I'm newly confident in learning / figuring out in a reasonable amount of time. It is much easier for me to say "yep, no big deal, I'm on it" when people are looking for someone to take on some ambiguous project using some toolset that nobody at the company is strong with. It solves the "blank page" issue with figuring out how to use unfamiliar-to-me-but-widely-used tools, and that is like a superpower, truly.
It's pretty decent for "happy path" test cases, but not that good at thinking of interesting edge or corner cases IME, which comprise the most useful tests at least at the unit level.
I'm pretty skeptical of #4. I would be way too fearful that it is doing that plain text to markdown transform wrong in important-but-non-obvious cases. But it depends on which quadrant you need with respect to Type I vs. Type II errors. I just never seem to be in the right quadrant to rely on this in my production projects.
The "really good intellisense" use cases #1-#3 also make up a "background radiation" of usefulness for me, but would not be nearly worth all the hype this stuff is getting if that were all it is good for.
I term this "low-entropy code". Copilot is great at writing heaps and heaps of low-entropy code.
The thing is, if you're not paid by LOC, and care about your system as a whole, you normally strive to get rid of code if possible (any code is liability), and make the rest of it high-entropy.
Today's terrific autocomplete is tomorrow's legacy shit you have to deal with.
This site has honestly been absolutely useless when discussing new technology now. No excitement, no curiosity. Just constantly crapping on anything new and lamenting that a brand new technology is not 100% perfect within a year of launch.
Remove "Hacker" from this site's name, because I see none of that spirit here anymore!
The trouble is, there's still nowhere better.
TBH I'm more surprised when I don't need to help it now. After about 3 times where it cycles between incorrect attempts I just do the job myself.
I disabled copilot since it consistently breaks my flow.
And I'm sure the reason for that is the garbage input. From time to time I have to perform quantitative code analyses in our so called enterprise repositories. And the results are shocking every time. I have found an extremely poor SQL code block to type cast many columns in hundreds of projects. It was simply copied again and again even though the casting was no longer necessary.
The training base should be sufficiently qualified (and StackOverflow ranking is obviously not enough).
But unfortunately it's probably too late for that now. Now inexperienced programmers are undercutting themselves with poor AI output as training input for the next generation of models.
Almost no products fit this description, and if they do then the marketing department is getting fired.
Does a Mcdonalds burger look like the picture?
If you go in with a healthy dose of cynicism IMO LLMs can impress. I’d call it a better google search and autocomplete on steroids.
Sometimes? But I don't go to McDonalds for the loss function between the picture and actual product. I go for the fast food and good taste (YMMV).
> If you go in with a healthy dose of cynicism IMO LLMs can impress.
I use them everyday in one way or another. But they're not replacing me coding today. Maybe tomorrow. And I go in with a healthy dose of optimism when I say this.
> I’d call it a better google search and autocomplete on steroids.
Sure, but this particular discussion is not about its ability to replace Google Search and / or Autocomplete.
It actually does in the countries that require it. You know you can write ACTUAL "truth in advertising" laws right?
I hack on them till I get something sort of satisfying.
> You'd be surprised how much better your output is using prompting techniques tailored for your goal.
The biggest problem I encounter is context length, not necessarily the output for small inputs. It starts forgetting very fast, whether it's Claude, GPT+ or other self hosted models I've tried.
The problem with this entire space is that we have VC hype for work that should ultimately still be being done in research labs.
Nearly all LLM results are completely mind blowing from a research perspective but still a long way from production ready for all but a small subset of problems.
The frustrating thing, as someone working in this space awhile, is that VCs want to see game changing products ship overnight. Teams working on the product facing end of these things are all being pushed insanely hard to ship. Most of those teams are solving problems never solved before, but given deadlines as though they are shipping CRUD web apps. The kicker is that despite many teams doing all of this, because the technology still isn't there, they still disappoint "leadership". I've personally seen teams working nights and weekends, implementing solutions to never before seen problems in a few weeks, and still getting a thumbs down when they cross the finish line.
To really solve novel problems with LLMs will take a large amount of research, experimentation and prototyping of ideas, but people funding this hype have no patience for that. I fear we'll get hit by a major AI winter when investors get bored, but we'll end up leaving a lot of value on the table simply because there wasn't enough focus and patience on making these incredible tools work.
...this is what happened in 99-2000. It took 3-7 years for the survivors to start making it usable and letting the general public adjust to a new user paradigm (online vs on PC).
Our jobs are safe. I would even expect more "beginners" to try something with AI and then need an actual programmer to help them
( At least, if they are unwilling to invest the time in development and debugging themselves
Ps. Probably all the given examples are in top 3 most popular programming languages.
Amateurs at Python, Fortran, C, C++ and all programming languages. Amateurs at car engineering, airplane engineering, submarine engineering etc. Amateurs at human biology, animal biology, insect biology and so on.
I don't know anyone who is an amateur at everything.
In submarine engineering, they will be able to design and construct it in some way like 3d print it, and the submarine will be able to move into the water for some time before it sinks. Yeah, probably for submarines a higher percent should be achieved before they are really useful.
No, and that is a one of their limitations. LLMs are human-level or above on some tasks - basically on what they were trained to do - generating text, and (at least at some level) grokking what is necessary to do a good job of that. But, they are at idiot level on many other tasks (not to overuse the example, but I just beat GPT-4 at tic-tac-toe since it failed to block my 2/3-complete winning line).
Things like translation and summarization are tasks that LLMs are well suited to, but these also expose the danger of their extremely patchy areas of competence (not just me saying this - Anthropic CEO recently acknowledged it too). How do you know that the translation is correct and not impacted by some of these areas of incompetence? How do you know that the plausible-looking summary is accurate and not similarly impacted?
LLMs are essentially by design ("predict next word" objective - they are statistical language models, not AI) cargo-cult technology - built to create stuff that looks like it was created by someone who actually understands it. Like (origin of term cargo-cult) the primitive tribe that builds a wooden airplane that looks to them close enough to the cargo plane that brings them gifts from the sky. Looking the same isn't always good enough.
Also take a look at rare diseases and doctors [1], in which machines are already better at diagnosing thousands of different rare diseases. Is is fair to say that machines are better at diagnosing diseases in general, just because they diagnose rare diseases, which each one, every doctor will need to diagnose once or twice in his career? Not clear at all.
Right now we are constrained by data, but that constraint will go away in 5 years or so. Will AGI will be achieved by then effortlessly? I have my doubts. My sentiment is that, even if AGI is never achieved, every small advancement in reasoning ability, in context window, in multimodal sensors and actuators, will have a very broad effect on jobs, on the economy and the way we are currently producing anything.
I also have two crypto-bro friends that are hyping it up without having anything to show for it. Which is why I'm sort of complaining about they hype surrounding it. I agree with your post to a large extent. This is not production ready technology. Maybe tomorrow.
These generally don't require advanced logic or thought, though they can require some moderate reasoning ability to summarize two slightly conflicting text extracts.
Lots of corporate work would be enhanced by better summarization, better information dissemination, and better text extraction. Most of it is pretty boring work, but there's a lot of it.
VC hypes seem to want to mostly focus on fantastical problems, though, which sound impressive at dinner parties but don't actually work well.
If you're a VC, do you want to talk about your investment in a company that finds discrepancies in invoices, or one that self-writes consumer iPhone apps?
Only one of those is actually doable today.
Here is what it wrote, didn't have enough tokens for the last 20% of the article though:
A Longstanding Partnership: The collaboration began in 2014 after the pro-Russian government was ousted in Ukraine. The CIA was initially cautious due to concerns about trust and provoking Russia.
Building Trust: Ukrainian intelligence officials gradually earned the CIA's trust by providing valuable intel, including on Russia's involvement in the downing of MH17 and election interference.
Hidden Network: The CIA secretly funded and equipped a network of 12 spy bases along the Ukrainian border used for intelligence gathering.
Training and Operations: The CIA trained Ukrainian special forces (Unit 2245) and intelligence officers (Operation Goldfish) for missions behind enemy lines.
Friction and Red Lines: The Ukrainians sometimes disregarded CIA restrictions on lethal operations, leading to tensions but not severing the partnership.
Current Importance: This intelligence network is now crucial for Ukraine's defense, providing critical intel on Russian troop movements and enabling long-range strikes.
This is an important lesson that all SWEs should take to heart. Nobody cares about your novel algorithm. Nobody cares about your high availability architecture. Nobody cares about your millisecond network latency optimizations. The only thing that anyone actually using your software cares about is "Does the screen with lights and colors make the right lights and colors that solve my problem when I click on it?". Anything short of that is yak shaving if your role is not pure academic R&D.
The thing is though that technical debt and feature creep (away from flexibility anticipated by original design) are real, and sometimes a rewrite or refactor is the right thing to do - necessary so that simple things remain simple to add, and to able to continue shipping fast. It just takes quite a bit of experience to know when to NOT rewrite/refactor and when to do it.
It may well turn out that LLMs are NOT the path to AGI. You can make them bigger and better, and address some of their shortcomings with various tweaks, but it seems that AGI requires online/continual learning which may prove impossible to retrofit onto a pre-trained transformer. Gradient descent may be the wrong tool for incremental learning.
I don't think pre-train for every update works - it's an incredibly slow and expensive way to learn, and the training data just isn't there. Where is the training data that could train it how to do every aspect of any job - the stuff that humans learn by experimentation and experience? The training data that is available via text (and video) is mostly artifacts - what someone created, not the thought process that went into creating it, and the failed experiments and pitfalls to avoid along the way, etc, etc.
It would be nice to have a generic college-graduate pre-trained AGI as a starting point, but then you need to take that and train it how to be a developer (starting at entry level, etc), or for whatever job you'd like it to do. It takes a human years to practice to get good at jobs like these, with many try-fail-rethink experiments every day. Imagine if each of those daily updates took 6 months and $100M to incorporate?! We really need genuine online learning where each generic graduate-level AGI instance can get on-the-job training and human feedback and update it's own "weights" continually.
If you know a little about the math behind gradient descent you can see that an embedding layer followed by a softmax layer gives you exactly the best Bayes estimate. If you want a bit of structure, like every word depends on previous n words, you get a convolutional RNN which is also well studied. These ideas are natural and elegant but maybe a better idea is to comprehend the research already done to avoid diving into dead ends too much.
Not sure what approaches you are considering as dead ends, but RNNs still have their place (e.g. Mamba), depending on what you are trying to achieve.
That being said, something is definitely coming. 50% correctness would probably be well worth using — simple copy/paste between my editor and GPT4 has been useful for me, and that's much less likely to completely solve an issue in one shot — and not only will small startups doing finetunes be grinding towards better results... The big labs will be too, and releasing improved foundation models that the startups can then continue finetuning. I don't think a new AI winter is on the horizon yet; Meta has plenty of reason to keep pushing out better stuff, both from a product perspective (glasses) and from an efficiency perspective (internal codegen), and OpenAI doesn't seem particularly at risk of stopping since Microsoft is using them both to batter Google on search (by having more people use ChatGPT for general question answering than using Google search), and to claw marketshare from Amazon in their cloud offerings. Similarly, some AI products have already found product/market fit; Midjourney bootstrapped from 0 to $200MM ARR (!) for example, purely on the basis of monthly subscriptions, by disrupting the stock image industry pretty convincingly.
Yeah, I remember speech recognition taking decades to improve, and being more of a novelty - not useful at all - even when it was at 95% accuracy (1 word in 20 wrong). It really had to get almost perfect until it was a time saver.
As far as coding goes, it'd be faster to write it yourself and get it right first time rather than have an LLM write it where you can't trust it and still have to check it yourself.
Not sure if I can agree. There would definitely be a value in looking at what libraries the solution uses, but otherwise it may be easier to write it oneself, especially when the mistakes are not humanlike.
A 13% success rate is both wildly impressive and also WAY below the level where I would personally find something like this useful. I can't even see reaching for a tool that I knew would fail 90% of the time, unless I was desperate and out of ideas.
And the nice thing about an AI here is that I think it will actually find a different subset of these tasks to be easy than a human would.
My experience of working with interns, or low-skill developers, is that the benefit normally flows one way. You are taking time out from completing the project to help them learn. Someone/something of low capability isn't going to be relieving you of the large or complex tasks that would actually be useful, and be a time saver - they are going to try to do the small/simple tasks you could have breezed through, and suck up a lot of your time having to find out and explain to them how they messed up. Of course Devin doesn't even have online learning, so he'd be making the same mistakes over and over.
Or for example if you commit todo's in your code, the ai will pick up on them and give you some options later on.
If the failure rate is 14%, just let it try a bunch of times. (half joking here)
The way I see it is at least the project issues are getting some attention, which is arguably better than no attention. If it can just fix simple things, at least you can focus on the complex things and not worry about postponing the low hanging fruits.
IE: Maybe rather than throwing it find problems, also use it to just build things from scratch - as an example:
The side hustle: make that a product:
Let someone like me who is not a coder have access to Devin for a month with the only goal of building a side hustle that brings a solo person a monthly income.
Then - Sell that so that the millions of people who have a solo idea and just need that "technical co-foumder" can use it to build. and limit it to one devin instance to a person to start...
I dont want it to do a one fell swoop - I'd like to say "build this module...
--
Have a contest where you get What can you build with Devin in TOPIC in 30 mins.
This would be useful in cases where test coverage is incomplete, maybe for auto-discovering/confirming bugs, and would really be needed if it's trying to fix bugs itself, especially if one dared let it commit bug fixes - would want to know that the fix worked and didn't break anything else (regression test - run other test cases too).
I highly doubt you're the dumb one here.
Yeah, AI will do the easy and fun jobs for you. You will only need to care difficult decisions that you're going to be responsible for. What a wonderful world...
I think people want it to completely replace developers so they can treat programming as a magic box, but it will probably mostly help big picture architecture devs compete with people who are really good at Leetcode type algorithm stuff.
The competition should be happening in the other direction.
I am going to work on something after this pot of coffee brews that I simply could not produce without chatGPT4. The ideas will be mine but the most of the code will be from chatGPT.
What is obvious is different skill sets are helped more than others with the addition of these tools.
I would even say it is all there in the language we use. If we are passing out "artificial intelligence" to people, the people who already have quite a bit of intelligence will be helped far less than those lacking in intelligence. Then combine that with the asymmetry of domains this artificial intelligence will help in.
It should be no surprise we see hugely varied opinions on its usefulness.
I'm quite wary of the long-term implications and downstream effects of that occurring at scale. AI is typically presented as "the human's hands are still on the wheel," but in reality I think we're handing the wheel over to the AI -- after all, what else would the endgame be? By definition, the more it can do without requiring human intervention, the "better" it is. Even if replacing people isn't the intention, I fail to see how any other effect could usurp that.
Assuming AI keeps developing as it has been, where will we be in 20 years? 50? Will anyone actually have the knowledge to evaluate the code it produces? Will it even matter?
Perhaps it's because Dune is in the air, but I'm really feeling the whole "in a time of increased technology, human capabilities matter more than ever" thing it portrays.
https://pavpanchekha.com/blog/chatgpt-herbie.html
> I wouldn't use it—sanity-checking its algebra is a lot of work, but even if you fixed that up, the high-level ideas typically aren't that good either.
This has been exactly my experience with chatgpt as well.
Rather, your “problem” is that you’re likely not writing uninteresting cookie cutter boilerplate that everyone can do and has done hundreds of times. The current crop of AI is cool for coding demos, not for solving real relevant problems.
> Just give me a product that works as advertised and I'll throw money your way
The people hyping this crap only care about the second part of that sentence. The first one is an afterthought.
It makes me question the truthfulness of the other claims.
They'd better have really advanced reasoning and planning capabilities way beyond everything that anyone else knows how to do with LLMs. There's a growing body of literature that leaves no doubt that LLMs can't reason and can't plan.
For a quick summary of some such results see:
He is one of a very small group of people (going back to 1989) to get a perfect raw score at the IoI, the olympiad for competitive programming.
https://stats.ioinformatics.org/people/2686
Glad to see that he's putting his (unbelievable) talents to use. To give you a sense, at the event where I met him, he solved 6 problems equivalent to Leetcode medium-to-hard problems in under 15 minutes (total), including reading the problems, implementing input parsing, debugging, and submitting the solutions.
I find that questionable. What does "software engineering in a business environment" require that a competent competitive programmer couldn't also learn?
Not saying competitive programming contest easy or something, but just pointing out that in a contest with timing constraint, the requirement realization phase cannot be fitted in.
Another analogy: martial art vs. military.
(This was so I could play populous, BBS to San Jose and get Grounded for a month for running the phone bill up to 926 for long distance calling into PCLink...
My dad yelled at me for playing video games "WHAT THAT EVER GUNNA DO"
--
years later I left my shift at Intel running the game lab.. to meet my dad for dinner.
He apologized to me for telling me that games and computers would never do anything.
I was touched he remembered.
---
These dumb olympiads will never amount to much, id bet.
If anyone is practicing for their B1 Dutch exam, feel free to use this link to get the practice paper.
https://usacognition--serve-s3-files.modal.run/attachments/4...
- deobfuscate complex requirements into well divided chunks
- find gaps or holes in requirements so that I have to write the minimal amount of code
- understand codebases so that the implementation fits nicely
I don't need an "AI software engineer", I need an "AI people person who gives me well defined tasks". Now sure, if you combine those two kinds of AIs I could perhaps become irrelevant.
No employment stability, no benefits, no health insurance, no vacations, at complete mercy of basically a single company.
Yeah, thriving.
I, in turn, suggest looking out of the window: modern carriage drivers are called truck drivers and their number is edging towards 10M worldwide.
Now add to this rail carriage drivers, sea carriage drivers and air carriage drivers. Years of progress changed the way you feed your "horse" and made reins somewhat more complex, but fundamentally nothing changed.
Aren't we all commenting on Devin's initial announcement published yesterday? Give it some time. It's a safe bet that a lot of companies which employ software engineers are currently tripping over themselves to get on the Devin waitlist, and the inevitable superior iterations on the idea of an LLM-powered dev that will absolutely be coming out probably starting later this week.
Why is that safe to say? You just said this was announced yesterday. I don't think Devin is standing out to anyone right now.
Its the AI people person who gives me well defined tasks
I'm curious what a large, mature codebase, with complex internals and legacy code looks like after you sick devin on it. Not pretty I suspect. In fact, I think it will become so difficult to fix that nobody -- neither human nor devin -- will be able to clean up the mess. By sheer volume, a broken ball of unfixable spaghetti.
I would be immensely pissed off if someone did this to an open source project of mine, or even to a closed-source codebase I'm working on. Not only would it not be useful, it would be moving backwards. Creating an icky vomit mess that we will probably have to spend years cleaning up after bug reports and complaints from customers begin mounting, and competitors can iterate faster.
Does that sound like something you want to deal with in your software business?
If you can tell if a solution is correct or not --- well, then you don't need to have AI write it for you.
I think AI programming can only work when the industry begin to treat "almost working" systems backed by human customer service as acceptable.
Did you just solve P=NP?
Many things are trivial to verify, but hard/time consuming to code up.
You probably shouldn't rely on this to write critical software, no matter the amount of manual QA you throw at it afterwards, but there is an abundance of non-critical use cases where you can quickly check if a solution is good enough for what you care about.
If they already know the answer then it’s probably more efficient if they write it themselves rather than having AI produce a potentially difficult to verify answer and try to verify it.
> When evaluated on the SWE-Bench benchmark, which asks an AI to resolve GitHub issues found in real-world open-source projects, Devin correctly resolves 13.86% of the issues unassisted, far exceeding the previous state-of-the-art model performance of 1.96% unassisted and 4.80% assisted.
While it is a progress, its far away from being useful to be a software engineer.
Budget-myopic executives already tried transfering that work to cheaper labor markets, but it worked much less than they expected and most ended up with unmaintainable software and loss of any hope for an actual engineering advantage against competitors. There's nothing new here.
There will be organizations that find a good and smart use for fully automated code generation, just like there is for outsourcing/offshoring, but it's not a universal win to just go with what's "cheaper" and organizations that don't look at the big picture are (as usual) trading short-term accounting gains for long-term value erosion.
So the actual value here is not "13% of all issues fixed for the cost of compute," but more like "a discount on human time for 13% of the issues". But you also have to factor in the time taken on the 87% of issues where leading you down a wrong path can be adding time versus human only. It's not clear to me how it all shakes out, and would require large-sample experiments with humans to determine. I would bet the final margins are small though.
Of all the SWEs out there that draw a salary, how many do you think would improve on this 14% unassisted figure?
"cars replaced horse drawn carriages. But we managed to adapt to that, the carriage drivers got new jobs."
My dudes. We are the HORSES being replaced in this scenario.
It's funny how a year ago I was really excited for how AI can help my everyday coding while fearful that it would replace me. Now I'm not really sure either will happen in the short term.
I always had some sympathy for people whose name becomes a product, but it was surreal to see the headline and realize it had happened to me.
At least I'm not named Karen. I'll think twice about how I use people's names in the future.
Maybe a silver lining is my name was attached to a clean and upstanding product. For the rest of you, maybe your name will be associated with the hottest new erotic fiction AI sometime soon.
Sure, one day we'll have AGI, and one day AGI will replace many jobs that can be done in front of a computer.
In the meantime, SOTA AI appears to be an airline chatbot that gets the company sued for lying to the customer. This is just basic question answering, and it can't even get that right. Would you trust it to write the autopilot code to fly the airplane? Maybe to write a tiny bit of it - just code up one function, perhaps?
I sure as hell wouldn't, and when it can be trusted to write one function that meets requirements and has no bugs, it's still going to be a LONG way before it can replace the job of the developers who were given a task of "write us an autopilot".
Most students who started jobs in software aren't writing software for aeroplanes. They're writing crud apps, doing third-party API integrations or doing basic debugging.
I worry that although not some specialized software devs but a lot will still have problems due to stuff like this.
I'm not talking today, but say 2-3 years down the line, who will have an intern when you can get an AI intern that can perform as the top percentile and comes at $20 per month subscription?
I don't think AI will replace jobs until it can do the entire job (full lifecycle from requirements to bug fixes, etc) and be interacted with in the same way a boss or team lead could interact with a developer. If you still need a person in the loop then it's not a person replacement - it's a productivity tool.
Even if I would have an AI assistant if I work on something I still have to understand that piece of work, I have to understand the requirements to give correct orders to the AI. I might become more efficient but I don't see an order of magnitude speed up where now I work on 5-8 projects which are in one ecosystem and I don't see suddenly switching to working on 50-80 projects where each 10 of them are different ecosystem that I also have to understand to make valid decisions - even if these are crud apps.
I wonder how much time of this was consumed by manually directing Devin into the right direction, manually fixing and undoing the mess Devin produced and watching Devin burn through $$$. As others said, being completely non-transparent about this burns a bit of trust, but I'd really like to know where we are right now. Since Devin is currently "invite only demos", a more realistic peek into the state of the art can be seen here: https://docs.sweep.dev/blogs/gpt-4-modification
My gut feeling (and limited experience): gpt-4 and other models are not quite there yet, but whoever prepares for the next generation of models now will eventually win big times. Or be replaced by simpler approaches.
People trying to use cars to pull horse carts are doomed to fail.
Trying to use AI to build the software of yesterday is a waste of time.
Can a user just talk to a computer and solve their problem regardless of the platform?
Seems like a no-brainer.
Maybe folks LIKE clicking on buttons and going through 10-step procedures to get tasks done.
Some mice like the maze more than the cheese, I guess.
For me, it's mostly information I want. I don't really need a full app for that. I want to know:
1) How much money do I have? 2) Did a check I cashed clear yet? 3) Are there any unusual charges? How is my spending this month? 4) Anything I should look into?
For actions I'd want to take:
1) Deposit check 2) Transfer money from account to account 3) Make a payment / authenticate with new EFT payee
All of that could be done conversationally, with a flexible level of detail. The data could be shown on any number of shared interfaces (messaging app, dedicated companion app, etc).
It doesn't obviate the need for software, and there are a ton of software ideas that exist today that will still be useful, as well as ideas that have yet to be discovered that will be useful. But...I expect the LOB app to go extinct in the next couple of decades.
Why on earth would you want that? Open the app, go to the account, enter the money with specificity, select the account to transfer it to, click the button.
Sure, being able to say "Transfer 200 to Steve" is nice and all but.. I just don't... consider it much better than the process we have today?
If I could actually do that verbally, that would be a huge improvement. Too bad it doesn’t work like that.
But, you're highlighting one of the challenges of changing the status quo. The "new thing" has to be significantly better than the old, otherwise people won't immediately want to switch.
Similar arguments were probably made when the iPhone removed the physical keyboard.
No engineering company worth their weight is going to build a world class technology business purely with generative AI in its current state. The risk in doing so currently is total and utter failure. I have a very hard time believing we're any where near that capability. Maybe your mom and pop startup could hire a prompt engineer to build a website and simple tool but we have yet to see those exercises surfaced to the mainstream; it's purely speculative.
I say, rest easy programmers. Your careers will be enriched more than axed with generative AI as a support tool for many years to come.
Also, if anyone who works in this field has a strong opposing belief, then consider OpenAI engineers are programming themselves out of a job which obviously is not the case.
https://news.ycombinator.com/item?id=36987454
Sweep is an open-source AI-powered junior developer
Instead it will mean that bosses can fire 75-90% of the (very expensive) engineers, with the ones who remain left to prompt the AI and clean up any mistakes/misunderstandings.
I guess this is the future. We've coded ourselves out of a job. People are smiling and celebrating this all - personally I find it kinda sad that we've basically put an end to software engineering as a career and put loads of people out of work. it is not just SWEs - it is impacting a lot of careers... I hope these researchers can sleep well at night because they're dooming huge swathes of people to unemployment.
Are we about to enter a software engineering winter? People will find new careers, no kids will learn to code since AI can do it all. We'll end up with a load of AI researchers being "the new SWEs", but relying on AI to implement everything? Maybe that will work and we'll have a virtuous circle of AIs making AI improvements and we'll never need engineers again? Or maybe we'll hit a wall and progress in comp sci will essentially stop?
Kind of like how art was supposed to be what humans would be doing while AI does the jobs we don't want, but looks like that will be the first thing to fall to the machines, while humans fight for carpet installation and plumbing jobs (for a while).
Me neither.
What people dream about is being able to go to a field and paint or sing at their leisure, not engaging in the soul crushing rat race that is artistic entrepreneurship.
AI is replacing the jobs that make money but are most certainly creatively bankrupt.
What is art? Apparently, anything can be art, so what's the difference between art and not_art?
From my personal experience, it is used more as a shitty excuse for substandard/lame/overpriced stuff to exist. And to boost it's creator's ego.
AI certainly isn't really doing it, just as people aren't.
Fuck art.
> Fuck art.
For me, art is both personal expression and interpretation, and yeah, those things can be imbued or ascribed to “anything”.
Even a comment on a web forum.
Your comment is an expression of your personality.
Your comment is your art.
Even with my nearly 15 years of experience, I'm not sure I see companies justifying the cost of employing me soon when someone half as "capable"* as me can work relentlessly and tirelessly at churning out half-baked features.
*I doubt AI agents will be able to use bigger picture foresight and reasoning (especially reasoning as software pertains to human user experiences) to architect sane applications (as we understand them today, at least) but this likely won't matter in a vast majority of cases.
Entry level positions in fields like medicine, law, the sciences, architecture, engineering etc can require years of intensive training before you're skilled enough to take on the role even at entry level.
Talented scribes did cool things with illustrations and flourishes that were lost in the transition, but ultimately spent quite a lot of their time stroking the letter "e" or "i" or whatever.
Meanwhile, typesetters found themselves in a whole new creative domain which was a different one than the scribes had been working in while also being informed by it and related to it. With their different workflow and this different domain, they were able to innovate in many ways unique to typesetting but also in ways that would circulate back to calligraphers and other inheritors of the hand-crafted letterwork tradition.
I'm personally not sold that we're soon to see LLM-based code generators replace software engineers anyway, but things are not so black and white as you suggest even were that to happen.
I think AI is hard to compare against the past. A printing press replaced some jobs, but AI could replace way more jobs.
Maybe it is like the printing press, but without typesetters.
They'll always be able to re-affirm their own preconceptions (fears) because its the "just so" fantasy future of their own making.
I don't see the point of coming to HN to trade those invented stories, as people here traditionally push to stay within the engineer's realm of how real things work, how they fit into the history of innovation, and what people might practically build with those things based on how they work.
There are countless speculative fiction communities better suited to idle "yeah, but what if the sky was purple tomorrow?" discussions.
More likely is that progress, discovery, and improvement behave more like dispersing bits of fog in an intractably large cloud that's always creeping back in on the clearings your made previously. You can sustain positive progress but its asymptotic at best and there are always regressions eating away at what you've done in the past.
So don't worry, there's always going to be more kinds of work to do, just like there's always going to be more physics to study.
Engineering muscle will be limited by chips instead of brains. Injecting millions of AI “engineers” into any economy would be a massive boon.
Many of us would be out of work, but if we can redistribute the AI-generated surplus, this will still be a net-win for us all.
I think the top engineers will still be able to find a niche regardless - we get paid to solve problems, and there will always be human problems.
We can't.
And any political willpower that exists is constantly used for smearing campaigns for either side of the political spectrum (e.g. look at them, they are the bad guys).
Automation tax is also the worst possible outcome, since it both stifles innovation AND doesn't address the redistribution problem.
which historical technological development that hugely improved productivity do you feel has led to an improved universal social safety net / public investment in whatever country you're from?
History suggests that we cannot.
This is very much like saying "but if we can redistribute the wealth of all the billionaires". Like, uh huh, but that won't happen because, short of fear of murder, people winning a class war have no incentive to stop.
No, it won't. Even during the vid, when people couldn't work, the politicians found a way to enrich the wealthy while only doing the bare minimum for those who lost their jobs.
The (first) French Revolution, the period in history to which all revolution is compared, did not distribute equality to the masses. It distributed The Terror, and then it distributed Emperor Napoleon, and then it distributed the Bourbon monarchy right back into power.
Then the (second) French Revolution changed one king for another.
Then the (third) French Revolution distributed another fucking Emperor Napoleon.
Revolutions aren't as equality-building as people want to believe.
I'm curious to know why you think there is even the slightest chance of this happening.
This won't happen. It does NOT trickle down, Mr. Reagan. The past 40+ years, every single graph you pull up shows the widening gulf between the wealthy and the not.
Let’s give all the shovelers spoons and live in the past where we can ignore the benefits technology can bring because we’re so afraid of what it means about the way we run society.
Considering we haven't seen talent rise to management, I'm not sure what makes you think that management will improve now.
I don't know which part of technology you work in but I can probably spin it as making possible for some manager to reduce staff.
There is a book called "Who Moved My Cheese?" and I wouldn't say its an amazing book, but the concept that not everything lasts forever, in relation to job security, is the takeaway.
Indeed it isn't. It's just that at each technological breakthrough, a number of (vocal) people thinks that it's a new type of revolution, ignoring those of the previous centuries /shrug.
If the goal with AGI is to literally take all of our jobs (all jobs in all sectors) then what are you suggesting we can do instead?
And furthermore, we’re not even remotely close to that. I guarantee this product actually sucks and is totally useless in practice. No offense, the tech is just not there yet. Saying this as someone at the cutting edge of AI and software development.
We continually hear anxiety about technology leading to mass unemployment and it keeps not happening. Instead, workers tend to have higher productivity, which drives higher wages.
Technological advancements transformed agriculture from being 75% of the workforce to being less than 5% of the workforce over the last 200 years and instead of mass unemployment, everyone found other ways to add value to society and it has been an absolute win, with higher standards of living.
(https://www.researchgate.net/figure/Percent-of-the-Labor-For...)
I thought the “The grapes of wrath” were exactly about the transient effects of the agricultural transformation.
Wages have not kept up with productivity.
This seems nice(ish) for someone like me with a senior title and experience (~15 years), but totally knocks the bottom out of the industry. How do new people enter the industry and become programmers like I did? What's the new entry point into software, and what does that role look like?
I can't tell if it would actually be good to knock the bottom out. If seniors become sort of like engineers going into the machine to tune things and build the big new things, who replaces them eventually? Will software get worse because of this?
One thing I wonder is if this will birth a new generation of software architecture which is essentially a complete mess which AIs can easily and efficiently manage due to being machines, which will require businesses to "take the leap". Basically you tell the system what you want and it'll generate it on a bespoke system using custom infrastructure which the AI is optimized to implement solutions on. The results might be amazing, but if something goes wrong, humans would have a hell of a time figuring it out. That wouldn't be a nice career for someone like me. It might still leave a category of old fashioned human-engineered software, though.
One thing about agriculture worth noting is that we had a LOT of other things we could work on. Agriculture held us back, so to speak. What is software holding back right now? If I'm unemployed by this advance, what would I do instead? I'm not sure how transferable my skills are. It could actually make me quite a bit less productive, not more.
If replaced, you essentially were the misallocated capital: capital paying your salary was inefficient compared to that same salary paying a more productive AI.
This means you will either have to find a way to become more efficient to compete against AI for developer jobs, or you will have to reskill. This reskilling period is what you're referring to as being less productive, but in the long run you will find a job and therefore will become productive again. We can't guarantee that job will pay as much as your former, but costs in the former greatly fell anyway, so it doesn't much matter.
In short: you will be reallocated and find your optimal productivity subject to your utility preferences through the reallocation.
I see a future where those displaced by automation are able to retrain with minimal cost or by charitable means. But this also means that current workers should be saving hard for a rainy day, as nobody knows when one may come.
"they ignore moral hazard"
Sometimes economic risks are negative, like spending your paycheck on lottery tickets, but other times they're positive, like starting a risky but ambitious startup or getting an education. If social welfare causes a significant change in the average person's risk tolerance, a claim that definitely requires substantiation, then I still might argue this isn't a net negative for the economy or society at large. Even if the claim is true, is using homelessness and privation as a threat (a threat often realized upon the unfortunate) a worthwhile ethical sacrifice? How much does it cost us not to do this, because I think it's worth some money.
"tend to produce worse outcomes"
If this is true, which I also wouldn't take for granted, then you'd still need to factor in the asymmetry in access that private charities have vs a social welfare system. Perhaps a person who gets the aid of charity benefits more than one who gets welfare, but does the average person?
"are less economically efficient"
This is likely to be true simply because government tends to be inefficient, but can be worth it for the above reasons and can be assessed objectively. I'd be fine with spending some margin more than should be spent, say 50%, if it means that 95% rather than 10% of people have access to food or housing or whatever else.
Overall, I don't see how charity can even serve as a mechanism for what we're talking about. If we see unprecedented unemployment due to AI, how exactly do we expect voluntary charity to expand to meet demand? What can we do other than some form of wealth redistribution if we don't want lots of people to starve? Furthermore, why can't we do both? If charity will expand to meet demand, then let's use social welfare to fill in the gaps and prevent destitution.
> I like this idea because I like liberty, and a rich person loses almost no agency by having some portion of their income or wealth taxed while a small stipend can make a huge difference in the choices available to a poor person.
I also like liberty, but to me the opposite of liberty is achieved when you redistribute for non public goods. Although you may have elevated the poorest, you've dampened the richests' purchasing power, however marginal. It is not pareto optimal, and to me the atomic unit is the individual and not the collective society.
> Do you have any evidentiary basis for the counter? Why would the system of voluntary charity suddenly improve its outcomes or have more money to spend?
"Crowd-out was small as a share of total New Deal spending (3%), but large as a share of church spending: our estimates suggest that church spending fell by 30% in response to the New Deal, and that government relief spending can explain virtually all of the decline in charitable church activity observed between 1933 and 1939."
https://www.nber.org/papers/w11332
This is an interesting problem due to its nature: good evidence can really only be collected once policies are in effect. Regression discontinuities around the New Deal are likely good candidates to study. The above paper estimates a 30% drop in religious charitable giving. I didn't look to see if they are able to discern whether this is due to the perception that the poor now get money from the gov so don't need extra or whether the income tax cut disposable income and thusly the charitable contribution budget.
Another instance is the Texas Seed Bill, where Grover Cleveland vetoed the disaster relief bill after finding no power enumerated to the federal government to provide aid. Private donations exceeded the Congressionally approved sum (or so I've heard but cannot find a source atm).
https://en.m.wikipedia.org/wiki/Texas_Seed_Bill
The logical basis is the following: by taxing someone you remove their purchasing power and thus naturally cut their ability to provide charitable contributions. Keep in mind the level of welfare we are already providing far exceeds what the wealthiest provide in their 50%+ tax rates. A lot of burden comes from "well to do but vunerable to economic shocks" folk, for which taxation rates have material impacts.
> If social welfare causes a significant change in the average person's risk tolerance, a claim that definitely requires substantiation, then I still might argue this isn't a net negative for the economy or society at large.
This is exactly why I mention moral hazard. We have to ask ourselves what the unintended consequences or the policies are, and how they may be abused.
Risk tolerance shifts due to free money will broadly impact the economy: you will necessarily increase demand as those with jobs suddenly find the opportunity cost of not working preferable. So you either end up with more unemployment or needing to tax at higher rates to meet the new demand for welfare. This might lead to increased economic output from the poorest but also may lead to less output from the richest. One thing is certain: risk is adjusted due to government subsidiaries/theft.
> Perhaps a person who gets the aid of charity benefits more than one who gets welfare, but does the average person?
How will you measure this? Are you quantifying strictly by the total dollars received by the needy? What if you flipped the script and asked whether the outcome is congruent to the expectations of the givers? E.g., are those dollars given charitably providing a better outcome for the givers than through dollars given by the charitable via taxation?
The benefit of the charitable approach is that there can be conditionals. "Do drugs and the money stops" kind of stuff. Society tends to think that is immoral for the government to do, but arguably the outcomes for the needy would be better if conditions were allowed. Charity also has the added benefit of being able to flow to local needs vs flow from the top down.
> If we see unprecedented unemployment due to AI, how exactly do we expect voluntary charity to expand to meet demand?
What is AI doing that will completely eridicate humans? I struggle to understand. Let's say you're displaced from programming. Could you become a farmer? Would machines undercut your price? Probably. But what if you're good at metal work and Joe is good at farming, and both of you have lost your jobs to AI? Maybe you decide to consume from only human, non AI based businesses. Suddenly, costs aside, people create an underlying economic network of non AI businesses and start to thrive again. This is essentially what we see with people trying to buy only from [insert preferences here] businesses.
I don't think AI is the scare people make it out to be. Ultimately, it will be a fun thing to play out so long as regulation is minimized.
Many never recover and age makes a difference.
In the short to medium term a bunch of people will be out of a job/career.
In the long term society may benefit overall.
On the other hand I’m not convinced humans are evolving fast enough to keep up with modern society. There are increasing rates of anxiety, depression, ADHD, etc, especially in young people. https://www.thecut.com/2016/03/for-80-years-young-americans-...
The only way that happens is if millions of people are significantly worse off. Most people can find work in the big economy, that doesn’t mean replacing a 100k/year job with a 25k/year job is equivalent.
Across a long enough time horizon technology tends to make most people better off, but in the short term it can seriously fuck people over.
And sure the long term trends reverse things as I mentioned, but it took 15 years to recover where it was in 2019. Most Americans are simply cut off from wider economic growth and recovery.
It is the case that productivity has grown with automation, but at the same time, median wages have stagnated, as the number of high-paying jobs have steadily shrank. Considering inflation, median income has been stagnant since about 1965. But productivity is at an all time high. But all the wealth that that generates is not going out into the world, its being concentrated.
I'm in automation with physical machines, and there's a part of me that sincerely hopes that the continuing automation of various jobs leads to a golden age where society's basic needs are always met by robots, and we're free to pursue our passions. But I'm honestly not optimistic that that will happen without a series of (likely bloody) revolutions and counter-revolutions until either the species is extinguished or our social system finally achieves a new equilibrium.AI can definitely be understood as an invasive species or a natural disaster in terms of the impact it has to our social ecosystem.
Now imagine that AI will make 20% people redundant over next 5 years - thats ~1.6 billion people.
BTW there is little AI makes "20% people redundant over next 5 years". I would bet $$$ against this.
1) technology development has tended to hugely improve society-wide productivity and be a general (though not unmitigated) good.
2) technology development has been absolutely shit for many individuals as their careers disappear.
people should be way more worried about good national governance and safety nets to deal with the terrible consequences of 2 while we reap the benefits of 1.
> Instead, workers tend to have higher productivity, which drives higher wages.
workers are capturing less and less of that, especially over the last twenty years.
hopefully whoever is in charge of your country has put a lot more thought into the future of work as automation comes for white collar workers.
A lot of these AI companies are promising the world and of course no one can deliver on that.
I think it's more likely we'll enter another AI winter first after all these impressive party tricks get boring and investors realize that just because a product uses AI doesn't mean it provides any value.
Also it seems like training and running these systems is incredibly costly and the prices AI companies charge for their products are being subsidized by investor money, which won't last forever.
If we were to pay for the actual costs at this point, I do wonder how many people would consider it worth the expense. But I wonder, how much should ChatGPT cost, for example?
AI engineers are coming, but I think we would have long retired before they reach a point where they can replace us.
That's why starting projects with something like Cursor makes you seem superhuman, but as the project grows the AI is more likely to get stuck because of previous low-quality choices. Just like with driving cars, it seems like you need strong supervision. (at least for the foreseeable future)
This is the same logic that has driven cheap off-shoring in non-technical companies.
For decades orgs have been able to buy "human-level" (i.e. humans) engineering for a tiny fraction of an engineer's salary, and there have been millions of eager salesmen for off-shore dev shops pushing them to do it too. After seeing the outcomes of this approach, I understand why well-paid engineers remain well paid. And why they'll remain well-paid after the LLM non-pocalypse.
If you think LLMs are so amazing, I would encourage you to see how much you can rely on them to replace human beings in real world scenarios. Not in contrived PR pieces and cherry picked examples but in situations where actual real people would otherwise be working together to deliver commercially valuable outcomes.
You believe you have domain specific insights that allow you to state, with confidence, that LLMs are able to replace a highly technical and well-compensated role at virtually no cost. If that's the case, you're sitting on a gold mine. If I believed that, I'd be starting a development agency tomorrow.
"If I believed that, I'd be starting a development agency tomorrow."
I see nothing in the original article that doesn't strike me as the techno-optimism of the 1960s where people made movies and books saying "It's the year 2003 and the humanity is exploring the vast depths of the Universe".
So again, it's a very plain old boring techno-optimism.
I am sure they can automate some work (like scaffold a certain CRUD part of the app) but there are always nuances and specifics and the current generation AI has so far proven inadequate in catching those and taking proper care of them.
This is a company spending investor money selling pickaxe hand grips during a gold rush.
For real evidence, look for companies selling engineering time much greater than the amount of their total engineer count who have good customer retention across projects.
Where's the app they built in an afternoon using Devin? Where's the software product that Devin actually built a month ago and was being used by thousands of people?
Their actual business seems to be closer to "Lets milk some of that sweet sweet high income from SWEs with FOMO about AI"
AI can't do 100% of the jobs, but it seems like we're somewhere past the 100x capabilities point. AI should be able to make a good employee able to do 100x the output at the same level of quality, and AI only gets more efficient and capable from here on out.
Lawyers doing highly specialized work like doc review can use million token context lengths to achieve days or weeks of work in minutes or hours. Doctors can review huge quantities of literature in search of information relevant to their patients, maximizing the value of their time. Any knowledge work, anything that has repeatable processes, anything not requiring physical work in the real world, will be subject to increasing, accelerating, and unstoppable automation. Market forces will reward efficiency, and eliminating human overhead with relative cost reductions approaching 99.99% is a victory for companies nimble enough to pull it off.
We're living in the apocryphal interesting times.
Lawyer A cares about productivity, and is paid by the hour, and so charges 1 hour of doc review at $4000, using the latest greatest technology, and is able to bill 1 hour of doc review to 100 clients. He begins to charge $3500/hr to undercut competition, and intersperses each week of work with training his paralegals and associates to amplify their capabilities.
Law Firm B doesn't care about productivity, and bills 1 hour of doc review at $4000 to 100 clients, but have to pay 100 lawyers in turn. Several weeks later, their clients have left for the cheaper, higher quality competition.
Being paid hourly doesn't mean you benefit from inefficiency. Small hungry shops who can rapidly embrace and integrate new technology are going to eat giant firms alive.
When you can set up agents to replace employees, consultants, contractors, and so on, even if the AI is only 90% as good as the average human, that's a staggering advantage for fast movers.
That is exactly what it means and what happened.
Lawyers are one of the last professions to move to the digital world.
For ages work was still done on paper and even right now most law firms don't use any non-LLM content search, autocompletes, content organisation, that could be considered "modern".
>Devin correctly resolves 13.86%* of the issues end-to-end... previous state-of-the-art of 1.96%."
While the cliff everybody will be shoved off is coming closer, it still seems engineers have plenty of room to do their job. Whether the next shove will come in months or years it is still early to tell.
It's important to remember that the reason AI can do it at all is that millions of software engineers wrote decent/good code and made it available on the internet. The reason it's going to be a winter is that going forward, there's going to be a huge reluctance to share code by people.
And that AI will never write code better than humans. It will write it 90% as well, still requiring a human to fix the last 10%.
It will further create a bimodal distribution of wages in the software industry. Those who know how to clean up the last 10%, and those who don't (AI prompt monkeys). The rift between these 2 categories will keep widening.
I mean, think about it. Rationally speaking, if people can hire AI engineers at a fraction of the cost of minimum wage employee, then the price pressure on software will be "significant" to put it mildly. The more complex a piece of software is, the more incentive there is for open-source developers to collaborate and pool their resources to make a cheaper alternative using AI agents. This logic could even extend to the AI agents themselves.
I know my prediction is deeply flawed, but basically, if we create an AI that's a drop-in replacement for (most) software engineers, we're probably going to have a massive deflationary crash not just from the software sector, but in the economy in general considering that many white-collar workers have also probably been automated, with the consequences that follow.
I might lose my job, but I'm going to take so much more down with me. My loss is their catastrophe. Sounds terrifying, maybe terrifying enough for our political representatives to address the elephant in the room, that an AI just took out their tax base, the people who still have a job probably make so little that their tax contributions are cancelled out by government benefits (not just welfare mind you, but the simple fact that they drive on free roads), and taxing corporations won't fix this when corporate revenue is probably going to also decrease.
Otherwise though, we're notoriously bad at predicting the future and estimating the general difficulty of something as ambitious as trying to replace certain careers with AI or robots. Any argument you could make for why software engineers (or white-collar work in general) will be different this time was probably made earlier by someone in reference to something like self-driving cars or physical labor, or possibly in reference to a prior attempt at automating white-collar work using a more primitive form of AI.
Best course of action, individually, is probably to skate to where the puck is heading and make an earnest effort at improving your productivity with software copilots, but otherwise I think the least likely outcome is that automating the software engineering profession is as easy as the e/acc crowd on Twitter would have you believe.
Perhaps a more realistic prediction of the future is that we start to identify a little more as designers, concerned more with the design of software and how it functions, rather than engineering concerns like making code performant and efficient.
I felt that software engineering as a career had a short shelf-life around 2020, but I wasn't too worried about it, as I figured I'd probably just transition to design as a career, I kind of think graphic design is cooler anyway. I began to have doubts about this plan once DALL-E caused an existential panic among illustrators, but otherwise I think it still has legs.
Those who adapt will probably do so by essentially becoming UI/UX designers who happen to also know how to code well, and probably know some other more general design skills just to round things out a little more.
Secondly, bosses try to fire the most expensive programmers ever since expensive programmers became a thing. All the outsourcing to wherever the salaries are much smaller is being attempted even as I type this now, by probably no less than 1000 companies, in this very second. Why hasn't the area in general still gotten rid of their expensive programmers?
Easy -- the outsourced "talent" produces crap that then costs more to fix and repair than if you hired proper programmers in the first place. Of course that requires some forward thinking that's not limited only by the short-sighted "we are about to save money muahahaha" mindset and as we know many businessmen are incapable of looking forward -- hence this mistake is being done 24/7.
It's 99% the same with these AI coding bots.
Wishing cost savings into existence so far hasn't worked.
Will some interns and smart people who coast on much-higher-than-local wages because they wrote some Python to import their boss' Excel spreadsheets and automatically generate other stuff, lose their jobs? Yes, that's very likely.
Will the senior programmer be replaced? It's not a zero chance, surely, and we already saw significant layoffs in most big US companies but (1) most of the world didn't over-hire during COVID and (2) it's unclear if senior devs were fired, or actual redundancies, and (3) well, most of the world isn't the USA.
So again, you are vastly overselling the current generation AI.
Whether or not the tech can actually live up to the hype doesn't mean execs/VCs/etc won't try to get there anyway. That will, at best, result in a ton of volatility, as those trying to utilize AI figure out that it actually can't do what they want, and then have to hire engineers again, etc...
I don't know. Who knows where this is heading.
This tech could free up software engineers to focus on more interesting things. But that’s also true every time there are layoffs. Those engineers they got rid of were free to focus on more interesting things, had the company wanted to utilize them that way. Instead, of course, they reduced headcount to reduce cost.
Same will happen with AI tools.
I'm not saying that's what will happen, nor your version. Things are never that simple.
But shouldn't we be more precautious? And take time to understand the shift and prepare as a society for this big change? Why rushing on something that would potentially negatively affect millions people life in the hope of productivity boost?
The outcome is not all good, not all bad. We need carefulthinking and planning.
Because they won't be bought be the displaced workers.
Twitter/X fired about that % even before the AI meme. Companies are way too bloated and it's bound to happen.
Currently, mainstream AI usage in coding is at the level of assistants and glorified autocomplete. Which is great (I use GitHub Copilot daily), but for us working in the space it's obvious that the impact will be much larger. Besides us (Pythagora), there's also Sweep (mentioned by others in the comments) and GPT Engineer who are tackling the same problem, I believe each with a slightly different angle.
Our thesis is that human in the loop is key. In coding, you can think of LLMs as a very eager junior developer who can easily read StackOverflow but doesn't really think twice before jumping to implementation. With guidance (a LOT in terms of internal prompts, and some by human) it can achieve spectacular results.
I see LLMs failing at coding daily (one of the "perks" of working in the space), and I'm incredibly bullish on this approach.
And I don't think it'll replace humans or junior engineers. As programmers, we've been "replacing" ourselves since the days of assembler that replaced direct machine coding. This is just another iteration of it.
(if you do want a sales pitch, here's one: https://twitter.com/senkorasic/status/1765769482985722267 )
Assembler replacing direct machine coding or C replacing assembler was a new higher level ruleset replacing an existing lower level ruleset. But that change was largely static, meaning that you could learn a concept of a mapping of C syntax -> generated assembler and rely on it being predictive of the outcome of your program with relative ease. This is what enabled you to "forget about" assembler and move on with C only.
With libraries and frameworks, we introduced another level of abstraction that introduced an element of dynamism since the underlying framework could change at any time. This caused a lot of frustration with many developers already.
Now, with AI, another layer of dynamism and an additional layer of "lack of precision" is introduced. What the machine figures out as a solution may change in each run, the AI itself may change, the libraries, frameworks and programming language that it uses may change - all at the same time.
This raises the question how any human being should afford the time to train themselves in all the underlying components and the AI so as to be a meaningful expert in the field. Given the speed of development in the AI field, this seems like a losing proposition for anyone to invest their time in. Much different from the direct machine coding to assembler analogy that you mentioned initially.
Why would would Agents need multiple turns and a framework like Pythagora in a (near-future) world with GPT4-level of capability and 10M+ token context?
Imagine GPT4 (or any other LLM) as a very eager but not very bright junior developer that just started to work with you. It's good, but it'll need a lot of situational management for it to not go wildly off the track.
What improved models will bring us in the future is making Pythagora and other such tools work better for large and more complex projects. The tipping point will come when for example you'll be able to load Pythagora in Pythagora and continue development. While we do build some auxiliary/external tools with Pythagora, the core is still handcrafted mostly by a human, and I'm pretty sure that's the case with other code gen tools as well.
Because it's better. This was the insight behind AlphaGo, AlphaCode and AlphaGeometry. In Go, trying to get the entire solution in one forward pass required a 1000x larger neural net to achieve the same performance according to Noam Brown.
Intelligence requires iteration, introspection, planning, sub-task breakdown, and trial and error with feedback from reality (error messages, print statements). These meta reasoning structures (whether hard-coded or, ideally, learned) will move LLMs closer to how humans reason.
At least it's a small team who will probably be shown up by bigger players in the market and go out of business.
Adding features was too hard to implement in my limited spare time, but was clearly possible. I would have needed some form of test running for UIs or CLIs, and I wasn't prepared to go that deep on a project I wasn't going to get much out of.
It was crude and overly specific to what I was trying to get it to do, but it worked well enough to convince me that someone smarter than me could make a capable and truly useful version of it that could actually impact the industry meaningfully.
The real world eval benchmark puts Claude 2 way ahead of GPT-4, which doesn't sound right.
I'm guessing it's a fine-tuning of some existing LLM model or API, but this largely seems to be an "agent" and UI that includes some SWE like workflow coding to allow more complex requests to be asked than just an LLM could provide.
I remain open minded about what’s next and at the rate things are changing, I wouldn’t rule anything out a priori for now.
When I was using chatGPT to help guide me through some coding tasks, I'd find it could create somewhat useful code, but where it fell down was that it would put things into variables which would be better put into a class. It is this structuring of a complete system which is important for any real software engineering, rather than just writing code.
Yup. It's hard to evaluate things based on the demo.
We're building something similar (with an open source core), and publish our examples for everyone to check out, warts and all: https://www.pythagora.ai/examples
Making false, grandiose claims like that burns a lot of trust.
Focus on execution and quality.
In addition, when using GPT-4, I use it only when I have new thoughts, so the GPU occupancy rate is low. I probably use less than 5 hours of GPU time each month. DEVIN is sort of like an intern working for you, so you would probably at least make it work 40hrs/week.
These difference in GPU usage would probably make DEVIN 10 times more expensive for the business model to be profitable, that is, if they are using the subscriber business model like GPT-4.
I don't think there are any other viable business model for DEVIN - for sure it cannot replace or even reduce the number of human programmer due to LLM's unreliable nature and the necessity of code verification.
Sure it is no senior architect but the trajectory is insane. Wasn’t that long ago that LLMs barely managed coherent poems. Now it’s troubleshooting code problems on its own?
Sure it’s just a gpt4 wrapper but that implies the same can be done with gpt5 and six etc.
Project it forward and that does actually become non trivial
My intuition is that we’re beginning to reach the bounds of what we can do with LLMs. All we’ve been doing is applying the technology to other mediums (images, video, code) and yeah we’ve been seeing some really awesome/terrifying results. It all looks like it’s 80% of the way there, from which the first thought is that in a couple of years it’ll be 100% of the way there and we’ll all be jobless.
This reminds me of the full self driving hype. The demos were so exciting and all that was left was getting these last couple edge cases worked out and then we’d all be in self-driving taxi pods….
Just let me try the goddamn product.
By the time you let me in, I don't care anymore or another competitor catched my attention already.
Neon, the Postgres as a service put me in such a long wait list that by the time they invited me in, I was already on a completely different solution (and was happy).
Side note: I'm kind of offended that something called 'Devin' is going to take my job. If you're going to replace me at least let me keep my dignity by naming it something cool like 'Sora'
The only difference is that there might be a little less profit incentive now. Perhaps we’ll get some interesting ideas from people who never would have been able to create them.
https://www.youtube.com/watch?v=dHlv7Jl3SFI
The real problem is coherence (logic and consistency over time) which is what these wrappers try to address. I believe AI could probably be trained to be a lot more coherent out of the box.. working with minimal wrapping.. that is the AI I worry about.
ok
I have tried using GPT4 & gemini extensively, and the amount of bullshit generated makes it unreliable if you don't already know the domain. These tools lack the critical stuff (being context-aware), and just make up libraries and APIs. Yet you can't be sure when it's bullshitting or not, making it an exercise in frustration for anything that's not trivial.
Save your money and buy an o'reilly subscription.
Namely, the client was asking for an unusually specific (for Upwork) ask. It was an almost perfect example of a job to be given to an AI agent for testing purposes.
I really want to see what this does with an open-source backlog. The more mundane (yet critical) of a project, the better.
What would be good projects to feed it? I suggested Mozilla and llama.cpp, but there's got to be something better as a use-case for it.
On a one by one basis I can use VSCode github copilot to rewrite each one the way I want it.
What I want to do is iterate through all functions in the files and do each one of them.
I know we are getting there, but does anybody know how that can be done right now?
One of the reasons I have stayed well clear of the IDE tools is they force me to use their own model.
While they might be convenient it means I can't switch to whatever the SOTA model of the day is at the drop of a hat.
Opus is awesome and well worth a shot.
But a software engineer absolutely can buy access to AI services.
I have no idea how this will end up, but it'll be different to before.
So many comments about how insufficient the tool is.
Our heads are really in the sand, I'm afraid.
Devin may need some additional help for awhile.
The hardware (or OS) is useless if it doesn't give good results, even if it theoretically could.
As someone passionate about the potential of AI in tech, I can't wait to see what amazing feats Devin will accomplish. And who knows, maybe one day, companies like Munesoft Technologies will reach similar heights with their own AI-driven advancements. Here's to a future filled with endless possibilities! #DevinAI
It's not 100% automated but saves a lot of time spent on writing code.
It works by composing prompts from tasks instructions, source code context and formatting instructions, resulting in high quality prompts that can be fed into LLMs to generate high quality code.
Business owners have been getting sexually aroused at the prospect of taking a developer's salary and putting it in their own pockets for decades. Each iteration of this wet dream has only locked businesses into "low code" systems that require even more, highly-specialized developers to operate. Right now, and probably for a while, Devon, et al. is on par with the drag-and-drop automagical app building snake oil stuff.
LLMs are useful to help developers be more productive, which does translate to lay-offs, but until someone creates an AI that can translate the absolute fevered gibberish that comes out of business people's heads into a profitable piece of software, this is just MS FrontPage v100.0.
Just like an entire industry sprang up around fixing WordPress websites that business owners thought they could do themselves, pretty soon we'll start seeing job postings for AI-Generated Spaghetti Unravellers.
I'm (half seriously) imagining a future were software engineers are mostly consultants that show up and talk with business folks, then talk with the local robot, and get the project to actually work. Bill $1k per hour.
https://docs.google.com/document/d/1byJgu1G_M58QVWmpZeEDthyA...
UBI is a pipe dream... it's not happening. The wealth and means of production won't be shared in any meaningful capacity. Wealth inequality can get a whole lot worse.
For every technological advancement, artisans are the first to be made obsolete.
Sure we have landfills full of unworn textiles, the market says its good, but overall, we keep destroying what allows humans to seek meaning.
Our governments and society have made it clear, if you don't produce value, you don't deserve dignity.
We have outsourced art to computers, so people who don't understand art can have it at their fingertips.
Now we're outsourcing engineering so those who don't understand it can have it done for cheap.
We hear stories of those who don't understand therapy suggesting AI can be a therapist, of those who don't understand medicine suggesting AI can replace a doctor.
What will be left? Where will we be? Just stranded without dignity or purpose, left to rot when we no longer produce value.
I ask this question often, with multiple contexts, but to what end? Who benefits from these advancements? The CEO and shareholders, sure, but just because something can be found for cheaper, doesn't mean it improves lives. Our clothes barely last a year, our shoes fall apart. Our devices come with pre-destined expiration dates.
Where will we be in the future? Those born into money can continue passing it around, a cargo cult for the numbers going up. But what about everyone else?
It's forcing the rich to give us UBI that is the problem.
It would be bare minimum to survive, if that. We'd need a complete restructuring of society to even approach dignity via UBI.
Start a local cooperative and build value from the bottom up
Consider how you can't really get by in most of America without a car because we designed our cities to require them. It would be a mistake to conclude that, because life is harder in a car-optimized society without a car, society must be better off optimizing for cars.
I'm not saying that things were sunshine and rainbows pre-industrialization, but there's some level of analysis to be done on the durability and value of a handcrafted piece of clothing, the care that goes into maintaining it, the value of a local economy, and the other side where you're forced to buy cheap items that degrade at a far faster rate.
If a town's local businesses are put out by a new walmart's ability to carry low prices, does the town truly come out ahead with those low prices? Or does Walmart simply extract more money from the town than it returns, leaving the town worse off?
Labor has never been to afford so much luxury as the modern day.
That said if you are satisfied with a pre industrial quality of life no one is stopping you from doing zero hours of work and just becoming homeless.
Where does that come from? I see it all the time. It's like there's zero ability to step outside the black and white. Either you must be totally onboard with the current system and not question the negative aspects, or you must live without dignity in the streets.
What I'm saying is we really don't need to be working 40+hrs/week in order to enjoy these advancements. People don't need to be made homeless in order for scientists who research these medicines to be paid peanuts.
Please describe the alternative. "Mass produced garbage" is exactly how we are able to feed the world. It's not some sort of conspiracy, just scarcity.
(Welfare and UBI is still capitalist.)
Nobody stops you from paying $1000 for a shirt made by artisans right now. Do you want to?
> Our governments and society have made it clear, if you don't produce value, you don't deserve dignity.
It's not somebody else who decided that. It's you.
If you don't do <insert extreme edge case> then your point is invalid. /s. Look, you dont have to pay $1000 for a shirt made by an artisan to get a quality shirt. It can be a bit more expensive, but much cheaper than that. And this can be true while being fair/reasonable to the artisan and also to the person acquiring it.
> It's not somebody else who decided that. It's you.
And what about my decisions? Or your decisions. Or anyone who is reading this's decisions? Surely the person you're replying to isn't some dictator who is deciding everything. It's not just the parent comment then. It's not just "It's you"
> If you don't do <insert extreme edge case> then your point is invalid. /s.
> Look, you dont have to pay $1000 for a shirt made by an artisan to get a quality shirt. It can be a bit more expensive, but much cheaper than that. And this can be true while being fair/reasonable to the artisan and also to the person acquiring it.
In the ideal case, this would be true: Following classical economics, there would be some predictable demand left as you go up the price curve, assuming genuine quality & labour.
However, this doesn't seem to be the case. The existence of low-cost mass-produced options leads to the satiation of demand above that spot of the curve. There still exists demand, but it's weirdly lower than predicted. People can shop down the price curve, but relatively fewer would do the opposite.
Aside from that I do think you make a good point and I'll need to think about it.
That was five years ago, with a recent re-sole this past summer.
This isn't an exaggeration by the way, but the cobbler who did the resole thanked me for bringing him the job, as it was a genuinely enjoyable experience for him. I assume most of his work is repairing suitcases going by the other clientele in the shop.
But sure, exaggerate that to be an artisan you need to be selling 1000 dollar shirts, rather than doing a 100 dollar service that doubles or triples the life of a decently made item.
> It's not somebody else who decided that. It's you.
No. The Republican National Convention cheered the idea of letting the poor and sick just die off in the streets. They cheered. Ron Paul asked "what are we supposed to do? Let our sick and poor just die cold in the streets?" and they cheered.
Jobs are sacred in the U.S. Job creators must be worshipped. Hard-working Americans are the lifeblood of yadda yadda. As soon as you don't have a job: fuck you, scum, you deserve to die in the streets. You are no longer of use to the wealthy, so you do not even have the dignity to sleep on benches or under bridges: they add spikes to any area you might find any comfort. Your children cannot receive an education. You get to disappear from view into some secluded slum until you die of the cold. It's not GP that decided that. It's tens (hundreds?) of millions of Americans who will cheer on your death if you lose your job. Most of which, of course, are a couple paychecks or a major illness away from being homeless themselves. Do not act like that's not a real thing.
The (very clear) point of Dr. Paul's RNC speech was that regular people are better at solving problems than the government. If the government stopped taking tax money to monopolize "helping the poor" (while wasting most of that money on ever ballooning bureaucracy) then churches and other private initiatives would step in to do the job better, and at better value for the money. We wouldn't just let our sick and our poor die in the street, because unlike a faceless bureaucratic machine that inevitably comes to exists only to perpetuate itself and its own power, we are humans who care about other people.
The people at the RNC were cheering for the goodness of regular people, for the fact that Americans give more to charity than any other nation, and for the positive libertarian vision championed by Dr. Paul.
When the government gets in the way, and makes everyone poorer (by excessive taxation and bureaucracy), however, people become colder, and they can always excuse it to themselves by pointing out that the government promised to solve the problem, and they already forced us to pay them to solve it. So now it is the government's responsibility, but a bureaucracy cannot have any feelings of responsibility, because it is a machine and not a human.
This is also what happens in communist countries like China and the USSR before it.
Unfortunately, many people will probably just read deceptive statements like feoren's, and never go listen to the actual speeches. The left loves to dissuade people from actually listening to conservatives. Just see how they are shamelessly lying right now about Trump's "bloodbath", hoping to stir up actual political violence from people who they manage to trick and frighten.
The left is always trying to use 1984 style newspeak to prevent people from having an honest debate, so naturally they talk as if it was an established fact that either a government bureaucracy must continuosly rob Peter to pay Paul, or everyone starves.
This is because the left wants as big and powerful of a state apparatus as possible, because they intend to use it as an army against their political enemies. A recent example of how the left sees the role of the government is Letita James' lawfare against Donald Trump.
Ron Paul's message was that whenever the government tries to "help", it always backfires. Expecting the government to solve social problems, and allowing them to take money from regular people, leads to financial ruin for the country, and it won't even fix the very problems it claims as the reasons why we must give up our freedom and prosperity. In fact it always manages to make the problems worse.
I shudder to think the direction this is all leading to.
True art will be rare and treasured, few artists will be rockstars. Till next fad we can't even see coming again mixes it all up.
But yeah these transitions do make tons of people miserable, losing jobs. Also overall middle class is disappearing, but thats the trend for quite some time. But it always paid off to anticipate what next generation of showels will be.
The reason I'm a pessimist is that I mostly see society preserving the status quo. I don't see AI democratizing things and freeing us, because we have fetishized the concepts of work and profit that we can't imagine a society that functions properly without demanding those two things be put above all else.
I don't envision a dystopia or a utopia, I envision a future where AI disenfranchises people who should be taken care of, bolsters profits of the already powerful, and replaces the most fulfilling human pursuits without actually saving people from unfulfilling toil, mostly because society will bend backwards to try to preserve the status quo.
Exactly zero of that unlocked time/value will come back to the workers.
It simply can't work within capitalism. We've seen it before so many times that it's a given. It's a natural conclusion to this.
Textile workers don't get happy when a machine can replace them despite it meaning they won't get injured anymore and this brutal job will get easier. It's objectively a good thing, but because our society ties all our livelihoods to work, they have to hate the machine.
Given how low standards are, executives will layoff swaths of people for a crappy AI very soon. I'm genuinely scared of how our world will cope with that.
I’ve decided the best thing I can do is vote with my wallet (i.e. as a consumer) and vote with my time (i.e. as a producer).
Consume commodities that are regenerative and products that are durable. Whenever possible, buy local to keep cash flows (wealth transfer) local.
Invest your time in contributing to products that encourage other people to do the same.
Yvon Chouniard said (I’m completely paraphrasing) something like “in the 21st century, the most powerful vehicle for transforming the world is the American corporation. Whether or not it should be, it is. If you think the world should be different, step 1 is starting a business”.
As a producer (engineer, for me), the best thing I can do is build tools to help people wean themselves off of Capital.
Buying (food, especially) locally means supporting a robust food chain that’s less dependent on refrigeration, global shipping, and a geographic arbitrage on agricultural labor. It’s better for the environment and connects you to your community and sense of place and seasons.
It means I can know what went into the food, in terms of soil, labor, and pesticides — something that I have no knowledge of at the grocery store.
A pound of apples, meat, or whatever from 1-2 counties away is more financially expensive, but if you can afford it there’s many non-financial benefits.
I’d rather purchase from a CSA than a national grocer, because that money mostly goes back into my local community.
You’re right the money doesn’t disappear when you spend it outside your locality, but it comes back in ways that are increasingly exploitative and/or depressing (chain stores offering no meaningful careers or opportunities for ownership, rent-seeking financialization, warehouses that bring noise pollution and packaging waste, etc.).
Again, if you can afford it, IDK why you wouldn’t prefer to spend the marginal dollar locally (i.e. private / family-owned businesses in a few counties’ radius).
It’s consumer choice and comparative advantage that actually lead people like me to shop locally...
They don't build 'em like they used to, but those of us in the rank and file are getting better off. We can't all be "I think I will just buy Hawaii" rich like Zuckerberg, but quality of life is, overall, improving.
I'm unsure why we default to extremes on that regard. A decent made T-Shirt should cost about $40, but right now the $40 shirt costs $.10 to make. Therein lies the self-consuming nature of how things are going right now.
Price does not necessarily imply quality (branding is much more of a factor).
Even if price did imply quality for certain brands, the question would be how much more qualitative? Will it hold 10x more wash cycles or last 10x longer?
In which case a consumer could make an actually informed decision on quality vs. price.
> we keep destroying what allows humans to seek meaning.
> Our governments and society have made it clear, if you don't produce value, you don't deserve dignity.
Those points are really interesting to me, I think you're right and I've never looked at it that way before.
Then the majority of us spent our time and energy in factories. That got automated during the Industrial Revolution. So we moved to service and knowledge jobs.
And now we're potentialy seeing the start of automation in those knowledge jobs. Will this be a painful revolution as the Industrial one? Where will we spend our time and energy on next?
> Where will we be in the future?
A German named Karl wrote a book with an answer to that question 150 years ago.
The main issue being, our governments and society have made it clear that the opinions of those who hold Capital are far more important than the unproductive surplus. Our entire society now is based around consuming cheap products we don't need, because the previous cheap products wore out prematurely. The focus on a profit motive tosses aside great discoveries that are unable to turn a profit, while encouraging useless advancements that only degrade our lives.
IE advertisements make money, but I've never met someone who believes their life to be improved by ads, except for a minority of shareholders.
On the other hand, there have been many medical advancements all but scrapped because the lack of profit outweighed the human benefit.
The profit motive is incompatible with a robust society. I won't pretend to have the answers on what should be done and how to implement it, but I know that our current system cannot last forever.
I hope that the current circumstances trigger a renewed interest in thinking about how we organise our society. It would be great if that could be done rationally, without taboos. I don't have high hopes though. The people in power want to stay in power, and any hint at socialistic or communistic ideas are quickly shot down.
Communism as implemented by Soviet Russia was pretty bad and failed, but that doesn't mean that the ideas of Karl Marx are all terrible.
I like the idea of a centrally managed economy but I don't think it's a stable system. There's too much opportunity for abuse by the central power.
I hope we explore the spectrum between extreme capitalism and extreme communism.
Does anyone have any reading recommendations that explore these kinds of economic and societal system?
For example, the "soviet" system was a bottom up democratic system where labor unions effectively formed the foundation. There was a lot of thought into how the system was formed and even the CIA admitted Stalin was more of a team captain than a dictator. Hell you can get a lot of interesting tidbits into declassified reports on the Soviet Union. There's the famous one about how the soviets were better fed than their American counterparts, despite the claims of breadlines.
It's really quite sad that we are basically not allowed to learn from the positive aspects of these projects.
For example, people hate on Stalin a lot, but his 5-year plans were so wildly productive and successful, even a young South Korea deep in a western backed dictatorship copied the idea in order to boot strap their industrial base. China continues to use this idea, the simple method of setting goals in the short, medium, and long term in order to use resources effectively.
you mean productive as in Holodomor?
By all means Op can say that they agree with some Marxist ideas, but when the real human costs of that model are very well proven out by now it should be a red flag if someone doesn’t acknowledge those costs from the jump. I’m not surprised this quickly went to suggesting Stalin wasn’t so bad.
You literally proved my point, One can't say "Stalin's 5-year plans were overall successful and copied by other countries" without "but holodomor". Yet you can say all you want about Churchill or Britain's policies without a smug "Bengal famine" appearing.
Premise is kinda all wrong, but IMO some interesting questions nonetheless are posed at the end of your comment.
First agricultural revolution was farming, that allowed for cities. Second agricultural revolution overlaps with the industrial revolution.
That was the transition from having only bio-mechanical energy available to also mechanical energy (the only source of power was no longer just muscles, but then things like steam power). The industrial revolution ended in the mid 1800s - a full century before the transition to service and knowledge jobs. Still, by the end of the second agricultural revolution, 50% of the labor force are still farmers. The industrial revolution was done decades before manufacturing labor participation peaked (in the US).
The industrial revolution was NOT the shift into factories, it was the advent of steam engine and steam power. You didn't have to pull a cart with a horse anymore, there were steam engines (trains). This is WAY before the model T comes out.
The idea that the majority of laborers hopped (in discrete steps) from farming, to factories, to service jobs is wrong. The percentage shift is smaller and more gradual than would be commonly believed, and plays over relatively long periods of time, most of those time periods playing out over just the last 400 years.
> Long ago most humans spent their time and energy on farming so they wouldn't starve. We automated that with a couple of Agricultural Revolutions.
This statements is misleading to the point of being incorrect.
This resource states there were three major agricultural revolutions [1]
Other resources refer to "The agricultural revolution", which was the second. "The Second Agricultural Revolution was both a contributing factor and consequence of the Industrial Revolution, which took place in the 1700s-1800s. As labor productivity and agricultural technology use increased and the population boomed due to a growing food supply, many people were left without land or work in rural areas. Thus, they migrated to the city, typically to find work in manufacturing." [1]
Note, by 1800 in the US, still 80% of the labor force worked as farmers. [2] Thus, there was a shedding from 80%+ to 80%, and many of those left for other work, namely cities. It's not the case though that the agricultural revolution caused humans to go from 90% farming to 10% farming, with everyone there going into manufacturing - it's much small numbers and much more gradual shift that occurred over a few centuries (notably, just the last 3)
The breakdown:
- first agricultural revolution allowed humans to farm and not be nomadic (12k years ago, I'm still shocked by how relatively recent this was)
- second agricultural revolution leveraged the industrial revolution, where a lot fewer people were needed for farming (but still a large number were required)
- third agriculatural revolution was in the 1900s and lead to much better crop yields
- arguably there is a 4th agricultural revolution underway today. I was talking to a farmer a year ago and they stated how amazing the tractors are now. They literally drive themselves & combine GPS data and automated controls to control the fertilization and watering down to the square foot.
Still today, 20% of the total global population is doing subsistence agriculture. [3] I would guess that a large fraction are still growing their own gardens for food supplementation.
> That got automated during the Industrial Revolution. So we moved to service and knowledge jobs.
"The Industrial Revolution was the transition from creating goods by hand to using machines. Its start and end are widely debated by scholars, but the period generally spanned from about 1760 to 1840" [4]
The industrial revolution is the transition from near exclusive muscle power (humans/oxen/horses) to steam power, machine power. From 1800 to 1860 the percentage of people (in the US) that were farmers went from 80% to 50%. It takes a lot longer before that percentage falls below 20%. The transition to service and knowledge jobs wasn't until the 1960s, that's about 200 years after the industrial revolution.
Still though, I can't find a single resource that shows US manufacturing percentage was ever greater than 30% of the workforce. The resources I find show there was a consistent increase in manufacturing output until essentially the late 1900s (speaking of the the US only), while at the same time there was a reduction in the percentage workforce in manufacturing.
> And now we're potentialy seeing the start of automation in those knowledge jobs.
Potentially is the operative word. "ThePrimeTime" aka "Primeagen" recently did a video on Devin: https://www.youtube.com/watch?v=80MPXoRHvK8 (start at 8:50, until about 9:45 mark)
His assessment, until Devin is 100x more powerful, two orders of magnitude better, only then will Devin even potentially start taking over jobs from humans. I generally agree with that take. I think we get there in maybe 20 to 40 years from now.
Personally, that idea seems similar to me as the idea that cars are imminently about to be self driving and you can lease your self driving car as a taxi during work-hours. That idea was floated by Musk since 2014, and every year since, that was going to be a reality within one year. [5]
--------------------------
I think your last questions though are very interesting, namely these: "Will this be a painful revolution as the Industrial one? Where will we spend our time and energy on next?"
Though, I find your premises to be inaccurate, and your timeline for when this next revolution is off by at least 30 years. It's yet to even start. I'll note we're still living through the automation of the service industry, that revolution is not yet done - computers have not yet eaten the world. As an example, the first successful 'pets.com' is still less than 20 years old, and that is just the act of automating a store-front to purchase cat food - plenty of examples of industries and services that are still largely paper based. A couple years ago I interviewed for a startup that was automating food chain supply, they noted that industry is still almost entirely paper based, and that is in 2021!
[1] https://populationeducation.org/a-timeline-of-the-three-majo...
[2] https://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=11...
[3] https://en.wikipedia.org/wiki/Subsistence_agriculture
[4] https://education.nationalgeographic.org/resource/industrial...
[5] https://futurism.com/video-elon-musk-promising-self-driving-...
Look at the unemployment rate history. Look at poverty rate history. Your argument is completely nonsense.
Perhaps dignity is seeking value that is scalable to entire humanity, hm? Isn't it dignified? If we cannot find value for fellow humans we can relax I guess?
> We have outsourced art to computers, so people who don't understand art
Ok, I am seeing a lot of bad faith arguments. Just because somebody doesn't have time to learn to craft and cruft of some very tedious discipline, doesn't really mean they don't understand it.
You don't need to throw rocks all day to get an understanding of physics. But people had to do it in the past, now we calculate, simulate and then just do one throw.
Also I am speaking as part time designer/artist. New AI tools are great, I could achieve similar results, but difficult bets were made, when i started. Now I just roll in images, some of them finetuned on my work, some are mix, some are completely nuanced. Many artists after initial sobbing(I was enthusiastic from getgo) started to learn and explore this tools.
People bring amazing things to life, that just were impossible with their limited resources (take a look, anime done with very small crew https://www.youtube.com/watch?v=tWZOEFvczzA).
I much dislike the mindset of "they took our jobs".
I really wanted to see what hacker news had to say on this :(
Currently these models don't provide an adequate enough confidence measure that prevents them from maximizing their potential. In the next few years we're going to reach a point where models will be able to tell if something is possible and avoid hallucinating, guaranteeing much better correctness. Something like that would be absolutely killer.
If you add on a top-down approach using a framework, such that it can architect a system down into small individual components, then that's a recipe for a really great workflow. The models we have now really shine in doing automated unit tests, and small bits of code to avoid limits with context size. Making the interfaces obvious enough, and being able to glue things together using obvious connections seems very possible.
I really do think that in the next few years we're going to see one of these tools really do well.