Post-GPT Computing
grady.io
grady.io
* Although ChatGPT is pretty good at generating code, it kept making simple mistakes such as calling non-existing APIs or introducing bugs. Some of them it could fix itself, some I had to fix.
* The code provided worked well for the "happy path" but failed miserably for some corner cases. I had to fix that manually.
* The code was working, but I wouldn't consider it production ready. It required some cleanup, unit tests, etc. Again, some of this with ChatGPT, some without.
* Not to mention that I was the one with the knowledge about the domain, what problem to solve, a vague idea of how...
Not to pick on OP but extracting a few seconds of video from a file is a pretty straightforward task, you can essentially do it with a bash one liner [1]. My biggest question is how ChatGPT performs with a large codebase, contributed over time by different authors, with complex domain logic and layers of abstraction.
I also had a brief existential crisis, but I just shrugged it off and got back to work.
[1] https://askubuntu.com/questions/59383/extract-part-of-a-vide...
Not being able to write code without it might be bad but it's a valuable resource and you should use it when it's available to you (for both)
Consider this classic: https://stackoverflow.com/questions/12122159/how-to-do-a-htt...
I have no doubt that ChatGPT will become even better than StackOverflow at answering questions. Is this really going to make us better programmers?
Yeah I think it definitely is, but I don't know why. Bing is better at looking things up (perhaps unsurprisingly) but Chat4 is better at creating things.
Bing produced incomplete code.
Only GPT-4 was close enough to be correct.
The availability of a button inside an IDE doesn't make this a fundamental change in how we work
Have you asked it to use any API that appeared after September 2021 (that's the cut off date for its data)?
Have you asked it to write code in less popular languages (e.g. Elixir)?
Have you asked it to write code for less popular or unavailable APIs (smart TV integrations)?
I also asked it to write using non existent but plausible sounding APIs, and it flat out says "As of my knowledge cutoff in September 2021, I have no knowledge ...."
Ae you talking about GPT4 or the default ChatGPT?
To quote GPT-4 paper:
"GPT-4 generally lacks knowledge of events that have occurred after the vast majority of its pre-training data cuts off in September 202110, and does not learn from its experience. It can sometimes make simple reasoning errors which do not seem to comport with competence across so many domains, or be overly gullible in accepting obviously false statements from a user. It can fail at hard problems the same way humans do, such as introducing security vulnerabilities into code it produces.
GPT-4 can also be confidently wrong in its predictions, not taking care to double-check work when it’s likely to make a mistake".
> I also asked it to write using non existent but plausible sounding APIs, and it flat out says "As of my knowledge cutoff
Ask it to write a deep integration with Samsung TV or Google Cast. My bet is that it will imagine non-existent APIs (as those APIs are partly unpopular and partly closed under NDAs)
"GPT-4 generally lacks knowledge of events that have occurred after the vast majority of its pre-training data cuts off in September 202110, and does not learn from its experience."
GPT-4 paper, page 10: https://arxiv.org/pdf/2303.08774.pdf
It feels like we are headed to a world where we can interact with a computer much more like they do in Star Trek; you ask the computer to do something using plain English, and then keep giving it refinements until you get what you want. Along the way, it is going to keep getting better and better and doing the common things asked, and will only need refinements for doing new things. Humans will get better at giving those refinements as the AI gets better at responding to them.
It is already incredibly good for being such a new technology, and will continue to rapidly improve.
Therefore the inverse can be safely inferred by nondisclosure.
I don't feel that my job is at risk of disappearing. Instead I think we'll be using LLMs as tools to do our job better.
I was discussing a bug with a colleague, so for curiosity's sake I decided to plug a similar question into ChatGPT. I was quite impressed with the solution it gave, and interestingly, it had the same subtle bug that our code had. What blew me away is that when I pointed out the bug, ChatGPT fixed the code by itself. On one hand I felt "phew, at least it needed me to point out the bug", but then I thought "I just (perhaps stupidly) provided training data so that down the road ChatGPT would get it right the first time."
ps.
ChatGPT: "You should only share information that you are comfortable with being stored or potentially used as training data."I don't see how people can see stuff like that and say "oh, it's just a fancy Markov chain generator" or "it can't reason". Even if that stuff is nominally true, how can people not be totally blown away by this? Just a couple years ago I think people would have been amazed that it can have totally natural, grammatically correct conversations. Moreover, for nearly 3/4 of a century the scientific community has pretty much coalesced on only using the output to define intelligence (aka the Turing Test). While I understand that ChatGPT may not 100% be there yet, I see no reason to believe that all this interaction people are having with it won't be fed back into it to drastically improve its responses over time.
It's more like the scientific community has spent the last 50+ years criticizing the Turing Test. Passing as a human is a nice engineering goal, but there has been a lot of doubt of using input/output behavior as the only measure of intelligence. If you took a basic AI class before machine learning became popular, the chances are the class spent more time on the criticism than on the test itself.
EDIT
And here is something else that just struck me. The errors in the code are obvious the minute you run the program, particularly in the beginning when the population dies off after day 1. Chat GPT is apparently incapable of running the simulation to check if its code actually makes sense. It needs someone to tell it. Think about that for a second. Can you imagine a fellow programmer handing you a piece of code without bothering to try to run it first?
I can see Chat GPT as an advanced assistant who can save a programmer a LOT of time right now, but definitely nothing more than that.
Also, I see lots of comments downplaying the potential impacts of LLMs because they can hallucinate or they have errors, and though I agree with all these points, I also want to shout "Gang, we're having a fully natural, back-and-forth conversation, with a computer. It speaks English and French and TypeScript!" This is stuff that seemed fully planted in the realm of science fiction only a decade or so ago. For example, I'm not sure I've ever seen ChatGPT make a grammatical mistake, or even generate code that can't compile (though I have seen ChatGPT "crap out" if the program it's writing gets too long).
I think I'm naturally a pretty skeptical person when it comes to tech hype, to the point of usually being over-conservative about potential impacts (I drastically underestimated the impact mobile would have on society, for example). But with ChatGPT, I feel like I need to take breaks just from the constant mind-blowing nature of it.
I saw a bunch of people talking about how GPT helped them code stuff on Twitter, so I thought I'd give it a try. Right now I'm building a sort of simple, mock version of the type of software that integrates with my company's APIs. I've successfully managed to create a simple web application that creates a new object, hits my company's API endpoint to create a corresponding object on our software, allows me to upload a document locally and then allows me to upload that document to our software via API as well. It's all a little messy and clearly not production-ready, but it works. It would've taken me probably a few months on nights and weekends do this (mostly refreshing myself on JS and Python). Instead I've done it in <24h (would've been shorter except for GPT-4's message limit).
I'm sometimes able to spot and fix GPT's bugs, but even when I'm not, it walks me through adding more logging and successfully debugs issues. Sometimes it takes a few tries and a little direction as to what I suspect the issue is, but so far it's fixed everything that's come up. I don't think this would be doable for a totally non-technical person, but I do think it'll get there pretty soon.
I'm just absolutely blown away.
I think we're going to see a lot of programmers who are going to trust GPT a little too much, and I think that's sort of scary. For the most part that is going to work out just fine. Often the quality of your programming isn't actually going to matter that much, because as long as it solves the business needs okish, then it's frankly great. That's not always the case, however, imagine someone using GPT to get your healthcare software wrong.
I'm still impressed with it in other areas. I think it'll do wonders in the world of office automation because it seems to have the ability to succeed at this much better than any previous "no-code" attempt where the logic would almost always end up requiring people who are basically programmers for it to work. I think GPT will help here, requiring less "superusers" for a department to move their data flows into automation. Especially in areas, where efficiency and stability aren't necessarily that important if the automation-tools mean you don't need three full time employees moving data from one system to another.
State your assumptions.
Read and summarize the pervious documents
generate a data flow diagram.
generate a data model.
Get it to inquire about use cases and requirements
generate tests for these uses cases and requirements.
Speaking of which: https://meta.stackoverflow.com/questions/421831/temporary-po...
> Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers.
> The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.
If we're lucky ChatGPT will poison itself by pissing in its well, but it will take a lot of good things with it.
Yeah, at this point I think this is a valid use case for GPT-4 in its current form. I would be comfortable using it to build internal process tools or standalone things like a simple browser extension. Nobody in engineering at my company would be dumb enough to let me start monkey around with our actual codebase though.
Maybe this is the chatGPT equivalent of "learning to google search properly". You got bad answers, but maybe someone more competent at chatGPT prompts and workflow would have gotten to a better solution more quickly, and we need to figure out what that means
There is also some research to show giving examples and answers sets up ChatGPT to give better results in the style of examples.
But overall, I have not seen anything that covers a process that would work most of the time.
People claim that AI can write code so they start firing programmers. Universities stop software engineering programs as there is no one taking the courses. People stop writing blogs or stackoverflow. Software engineers either move to other fields or start living offgrid. No new innovation or new line of code written by human.
Meanwhile, software quality get worse by each passing day and there’s no one to fix. AI poisons it’s own well by generating shitty code and now even simple tasks are taking 30 seconds. People say, “In the good old days, we used to get response in under 1 second”. Just like how they talk about cars and their durability in the good’ol days.
There is already so far beyond enough data available online for an AI to be a super-human senior software engineer/architect/whatever. Because there's enough for the humans, and an AI can do it better in theory. It just needs to be a good enough model.
Barring the most extreme edge cases, it's never going to be worse than it is right now.
And because of the corp greed. Cheap solution will always triumph good solution, so I am pretty confident that they will remove the guardrails way sooner that they should. I mean Facebook literally fanned the flames in Myanmar[1] so my bet is on corp greed this time too.
1. https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb...
It's already becoming a strain to review ChatGPT code committed by others, who try to save time by using it instead of thinking for a moment and then coming up with a proper solution to a problem.
IMO PM's are the clueless bunch of the whole set up. Often they are just making up things as they go and making it right through 'authority'.
PM's remind of me of early days of Sudoku craze when people thought they can put in any numbers in any squares and then later on puzzle would automagically rearrange itself to help them win. Backtracking never clicked for such people.
Once you are beyond simple toy apps, and have to build large engineering artifacts in iterations and realise it can't be from scratch in every iteration. The real fun begins.
You mean in areas you are less expert in? :) Maybe it's bad at everything, but each person can only tell when it comes to their own area of expertise.
Modern languages (and tools like autocomplete) have already helped that a lot compared to assembly code or binary, this looks like the biggest jump in a long time. The path of programming so far has been moving from "describe how to do something" to "describe what to do" which this is certainly in line with.
It’s easy to get amazed by something that can halfway do something you can’t do at all automatically. But as others have pointed out, it’s not that great at it and not knowing enough to do it yourself means you don’t know enough to catch and fix bugs.
So this move to using chatgpt and similar in production by people who otherwise wouldn’t be able to do things in production is worrisome, imo.
This quote highlights the challenges of accepting new information or ideas when they might jeopardize one's livelihood or status quo.
And afterwards it cleans the cup and puts it back in the cabinet.
If you want it to solve arbitrarily complex problems, you need to set up some sort of loop. People are already feeding the outputs back in as input in various primitive ways, but I suspect the real breakthrough will come when someone trains some sort of recursive transformer from scratch. (Assuming the current networks waste neurons in unrolling loops, we might possibly even see smaller models).
[0] Try the following family of prompts: "_ is an example of _, which is an example of _, which is an example of _...." etc to a depth of your choosing. At some point it bottoms out and you can't get any more levels out of it.
Yesterday I got a complex data structure out of it in 1h that we'd been talking about but not implementing because it would have taken a couple of days to get right.
In all cases it made mistakes and I had to rely on my experience as an engineer to ask the right questions and fix things. But god damn it made me insanely more productive.
Don't shrug this off and go back to work. You'll get left behind, and may not have a work to go back to.
This is supposed to take programming jobs?
HN is incredible.
Internal tools that automate 3 workflows we'd been doing manually. 2 node scripts and a super simple web app exposed on our private network.
The other day I was working with the Cisco Meraki API...I knew exactly what the script needed to do, but the calls were tedious and I didn't feel like learning the names of all the JSON columns, so I just had ChatGPT do it. I had to fix a couple mistakes, but the 20 minutes it took was better than having to read all the documentation.
Let's wait for "And now... long term memory is all you need" paper.
Also what happens to Europe? All these companies behind LLMs are from US, and Europe is nowhere to be found. This seems like it will dramatically accelerate the wealth different between the US and the EU.
And so on. Maybe the answer is in fact "yes", or even "yes, and it would have done these things even better than humans did". But so far it seems to be amazingly good at doing things that we showed it how to do.
If we stop creating actually new things, will it do that for us also?
Why would it care to do so? What interest does it have in creating new things on its own?
If the amount of people that will have social mobility opportunities will be equivalent to the amount of people who could have invented the MPG format or something comparable, then my point is made.
This is my insurance against LLMs, only works if the market demands new things...
The question isn't really about "genuinely new things". The number of permutations of existing things is such that at any given job you're likely to do old things in a new way.
E.g. you'd think that all streaming services are the same. Superficially, yes. Internally, Netflix, Disney+ and Apple Tv+ are likely to be different as night and day.
Could you?
But let's say that an MPG format specification is available. Even other code examples of interacting with an MPG file. But no examples, no library, no documentation on specifically how to extract a subset of one file into another file.
I would think a competent programmer could figure that out. Perhaps an AI tool could also; I have not yet seen an example of it doing so, but perhaps it could.
Of the handful of questions I asked, this might be the least interesting one. More generally, can AI tools advance the state of the art?
Would you?
If the technology pans out the way the techno-enthusiasts hope it will, upward social mobility will be nearly eliminated... unless there's some kind of successful Luddite revolution against the technology and the people that own it. But that's not going to happen: there are all kinds of social pressure against revolution, as well as strict gun control in most places. Anyone who tries to resist their obsolescence will soon find themselves either ridiculed and condemned or in jail.
Of course, downward social mobility will accelerate, and be celebrated by idiot technologists who just want to build tech, and don't really care to think about the consequences of the technologies they build on real people.
Those round to the same number: 0.
How is this not a huge problem? The vast majority of people are not exceptional. Cutting out that middle band of ability and resources is a surefire recipe for social unrest.
I don't know, the US has been pushing that envelope for 40+ years and people are still paying taxes...
And people can push the "meth consumption" envelope for years before they finally die from it, too.
Getting away with unsustainable practices for X amount of time doesn't prove they're sustainable and won't end in collapse. It just means collapse can take more than X amount of time.
To claim AI tools will strip out the wealth of the middle class and cause unrest can be disproved by the fact that it was beaten to it by the US corporations by over a generation.
The "middle class" was an invention of post war US to reward the soldiers who fought in Europe and the Pacific. Things are just reverting to a more long (as in multiple centuries) trend behavior of economies. Few haves, lots of have-nots and barely anything in between.
Ideally, defund the police and so on, so that every state worker also is keen on getting that wealth redistribution done.
I'm not underestimating civilians. If what you're suggesting was at all realistic, China would be a democracy and Trump would still be president.
Sure, tens of millions of unarmed people with a single mind could probably do anything (like a mass of zombies can), but you'll never actually get that. There are numerous mechanisms preventing such a mass from forming, and more to dismantle and negate it afterwards.
I'm not talking about right now.
> And trump is simply an idiot that only get enough votes somehow because both parties in the US are a joke to begin.
Trump literally had "a crowd of people at some point ... walk into the [government's] home and make [them] agree with handing over [power]." How did that go?
And at the very end, it will reduce to capital only, with no need for labor at all. Most people will be unemployed, and whatever capital they've amassed is unlikely to be enough to sustain themselves and their families for the long term. They (you) will end up as little more as impotent ants to AI-fueled Elon Musks, neglected until the infestation needs to be cleared to make way for some project.
I don't think it's that unrealistic. The trick will be, not going too fast, managing a few separate transitions, and making sure capital maintains control of the institutions with the monopoly on the use of force. The masses don't tend to act to project their interests until it's too late.
The masses are already showing signs of restlessness, and the only real problem right now is wealth inequality. Actual unemployment rates remain low. Forward in time a little, let's say 20% unemployment due to AI. The only way anybody is going to maintain their monopoly on use of force is if they hire every one of those 20% to be police. Right now the ratio of police to citizens is really low, and the ratio of weapons to civilians really high. I don't think the masses will wait all that long.
> The masses are already showing signs of restlessness
IMHO, "restlessness" doesn't mean anything. It would be expected in a AI-driven usurpation of labor. People have already been restless for decades due to de-industrialization, and that mainly got us Trump and an opioids, but the factories are still gone.
The key to fucking over the masses is making sure the "restlessness" doesn't get too strong, and doesn't have a clear (and correct!) villain identified, and maintaining a sense of inevitability.
> I don't think the masses will wait all that long.
IMHO, they probably will. Any individual or small group who takes action will be pilloried as wackos and thrown in jail. A larger movement will be (rightly) characterized as an insurrection and dealt with harshly.
People are complacent, and often don't realize they're really losing something until it's already slipped from their fingers.
I also think the Western world lacks the ideological tools to stop technologies like this. They'd basically have to start looking at technology like Amish do: rejecting technology that would undermine their social structure, rather than expecting the social structure to adapt to the technology.
We may not be that far away from when energy-intensive, latency-insensitive computing tasks are best located in space, to take advantage of cheap continuous solar power. The power capabilities of the next gen Starlink satellites are impressively cheap.
It does make sense, but you're not thinking about it clearly because you're too tied up in existing social structures. The end state "AI-fueled Elon Musks" (note that's a type, not a particular man) don't need common-man customers or their money, because they don't need to pay labor to operate their capital. They can directly operate their capital themselves, so they'll just do whatever the heck they want and nearly everyone who's now an employee becomes an ant.
At that point the main economy would mainly consist of billionaire ego projects and some trade between large corporations to support them. Common people would scrape by on billionaire largess and by squatting on resources not currently needed by billionaire ego projects and using it for small-scale subsistence production.
As you might have noticed, the AI boom will decimate the code writing jobs as well, something that the EU is behind on. Europe missed the "tech" age, but notice how the EU is not any poorer than the USA. Sure, some countries are poorer than others, but not everywhere in the US is Silicon Valley. Why? Because despite the EU missing out on "tech", actually the EU is very technologically advanced. Tech doesn't mean only low-touch high-scale computer-based businesses. There are chemists, biologists, anthropologists out there who don't know how to write a single line of JS and are paid like 1/5th of a junior JS developer, but the work they do is very valuable to society. Guess they don't need to learn JS anymore.
Also, notice how despite the thousands of layoffs, the US job data keeps coming out very positive - there's no unemployment problem. This is because of the markets, but AI will have similar effects. The world no longer needs that many CSS experts and React gurus who pull in $200K; the world apparently needs more hard-tech engineers and retail workers.
The AI thingy is devastating just for a subset of the "tech" workers and creative industries. It will enable other types of people and industries.
Startups who are trying to solve food production issues, for example, might finally outshine the next grocery delivery startup.
EU is significantly poorer than the US. Lots of different ways to measure it, but it’s a factor of roughly 1.5-2x in purchasing power parity.
Not saying the system isn't bad, but 10k for a doctor's visit is kind of a stretch...
Healthcare spending per capita in the USA is 12K, the second most expensive is Germany and it’s 7K.
The life expectancy in Germany is 82 and in the USA is 77, so it’s not the case that Americans are getting much better one, explaining the higher costs.
Oh and that appendicitis? It kills much more Americans than Germans(0.08 vs 0.06 per 100K).
I'm just saying that I see these weird takes about Americans paying like 10k for a single visit when that's just not the reality for most people. Only 8.6% of Americans were uninsured in 2020. Most insurance plans have a cap that doesn't even allow you to pay more than like 2-10k in an entire year.
Is it more expensive? Duh. It's just that there's more nuance to the issue. If I wasn't American (or I was an American that didn't understand how insurance actually works) I might have read that and believed that the poor Americans are all living in hell.
So yeah...we can talk about the issue because it's a big deal. In fact, I think it's the #1 biggest problem with American society right now and it's maddening that I never see anyone focusing on it. However, I don't like to see this kind of hyperbole from OP because people turn their brains off and stop looking at the facts.
That 10k doctor is a myth and certainly not something the 100k developer will have to pay. That's covered by his company. Healthcare is an issue in US when you're at the bottom of the food chain.
I'm just waiting for an "Ask HN: What are some job alternatives for people who know programming and can't get a job anymore since ChatGPT replaced us?"
Those businesses would not be around for very long, so who cares?
Europe in itself is (together) the single biggest market/economy in the world by the way, and the US is actually falling behind into developing-country territory when you look at the population and their access to basic services. And just because right now it is convenient to rely on the US companies, and we're deep allies btw, doesn't mean europeans couldn't spin up the same tech if really needed.
And given the amount of time needed to actually build this against the current pace of AI progress, the angry mob should materialize itself much earlier. French Nobles didn’t see the Guillotine coming either, even like a day before the revolution
At a point when all labor is obsolete there will be literally no method of survival for anyone who doesn't own the "compute capital". The two options will be to let everyone starve because they weren't lucky enough to shareholders in the company that owns all the bots, or just make that enterprise socially-owned and pay the unemployed workers.
How about subsistence farming?
Truthfully I think we'd have a few large societal shifts before we ever got to the stage where genius level AI could be spun up and down like containers, but it helps to illustrate the point that a post-labor society is incompatible with the tenets of capitalism, which is something that a lot of people fail to comprehend when they worry about AI.
The fact that the net result is positive doesn't mean that everyone profits equally. Having lived in capitalistic societies should have made that clear already.
Yet here we are.
It’s a human-political issue, it is not a technology issue.
What’s the difference now?
OK, put your money where your mouth is and send me 10% of your pay check.
And while we are here obviously I do my best to pay as little taxes as possible, but due to where I live I do end up paying more than 30% of my salary in taxes.
Did you ever put beers into those fridges? Or just took them? Because that's what looting is.
I guess Socialism is always nicer when you see yourself on the receiving end. We are both in the top 1% of the world, so we'd be giving away pretty much all we have.
You have no idea how I live to make claims about my political inclinations.
Then again I never advocated socialism, and you're fighting a shadow.
This is exactly the opposite of what you would expect given the increased efficiencies that come from adopting computer systems and automation.
I see AI as a continuation of this trend and I don’t expect it to put people out of work, bureaucracy will always find new ways to justify itself.
Like, I could technically have a newbie running commands on a production router for a script that I wrote out...but even if I let them do that there's no way I wouldn't at least supervise. I don't think most companies are even remotely comfortable with the idea of having an AI system running code on their systems no matter how smart it is.
So far technology has enabled use to increase economic output which means rising standards of living. Even if 99% of people subsist from selling their labor, the tools they use are a force multiplier that (in theory) drives wages up.
When you can spin up a bunch of Von Neumann level intelligence LLM-powered agents and have them run your company for you, there is no more labor to sell. You can either pay the former laborers to exist, or just let them starve.
So our two options are social ownership of all AI capital, or letting everyone without AI capital die, and let a handful of people live in the resulting AI-powered society.
The fact is that anyone who understands even at a basic level what the computer is actually doing and isn’t afraid to look at it at a low level can’t be replaced by an AI trained on stack overflow.
It may be that I will spend more of my time on code review of LLM generated code, or make my money in the new kinds of legacy code created by copy pasting ChatGPT snippets together instead of SEO optimized stack overflow scrapes.
For me the outcome is the same. The skills I need to be more effective than the machine are the exact same as they were decade, century or even millennium ago. I still don’t see these LLMs do any synthesis of knowledge, and they don’t seem to have a grasp of logic or grammar at the level I expect a bright middle school student to have.
Lol, I was thinking about this the other day. Eventually most devs will essentially just be praying to the Machine spirit to make the computer do what they want. A small few high clerics will bother to learn how computers actually work. The rest will simply be cargo culting to the maximum extent possible.
Same as it ever was.
People who are in tech just to "climb social ladder" i.e. only for the pay check are going to be pushed out by LLMs and people who are actually passionate about tech will remain. This will cause less and less shitty code to be written (of course for next few years even more bloated shit code will be written with ChatGPT and Copilot by noobs who have no idea what they are doing)
If you are a software engineer, this will output your productivity ten fold on the upcoming years. Now you don’t need to hire junior devs and can just build the product of your dreams with very limited capital.
In my opinion this technology will be as democratising as the YouTube’s early days.
Instead of worrying, learn to work with it. It will be harder for large companies/large teams to extract value from this compared to small companies/small teams.
It means competition between companies will increase but it isn’t necessarily bad for existing software engineers, especially solo founders.
You are overestimating the vast amount of "software engineers" in the world. The overwhelming majority of us are just programmers, we are just gluing together CRUD spaghetti in the random language we grew up with. We don't care too much about work or a career. And most of us don't want to do more, we want to get a decent salary for our boring work. And we certainly do not want to be "solo founders", build products of our dreams or increase our productivity.
This way of living feels threatened now.
At this stage, the best advice I could formulate would be to learn LangChain and prompt engineering, but these too are fast moving targets, and who knows what's going to be relevant in 2024?
I think the best thing one can do is learn how LLMs work, acquaint themselves with real implementations of it (ChatGPT, copilot), and then find ways to integrate these techniques into their companies.
Instead, look at the job postings for titles you want. Note the skills in demand at more than one job. Focus on those skills. There's your set of skills the market currently is in demand of.
Don't pivot your career. Don't burn the boat and jump into AI. Just be aware of these tools and get good at what they're poor at.
While I agree with the sentiment, there is way too much noise in that channel. Job listings written by non-technical people just throwing key words together, recruiters detached from specific roles and companies trying signalling growth to mention just a few sources of confusion...
I scratched and clawed, read tons of books, blogs, spent extra time polishing features beyond what was needed so I could learn new skills... but now I am a father of two young kids, with a wife. How long am I supposed to put in all this extra work? I'm likely slightly above average intelligence, but I'm far from being at the level where I could be an AI researcher... if I am even capable of doing the kind of math required there, it would require many years of learning.
GPT4 isn't going to replace me, but watching this space unfold really has me worrying about the versions that come out over the next 2-5 years.
A human is only so moldable, and while I am more than happy to learn new skills, I have no idea where to even start. What profession is safe? Where will the growth be in a field that will have equivalent or even near equivalent earning potential?
If GPT ends up getting to the point where it can replace me at my job, I really have a hard time thinking of a career path I could get into at this stage in life. It would need to be able to architect systems at a high level, write code to implement various features, communicate with stakeholders, document design decisions... if it can do that, it can do a whole hell of a lot of other jobs too.
Once it gets to that point, I don't think physical jobs will be that far behind on being automated either. We already have robots of all shapes and sizes (including bipedal), the main thing slowing down their deployment is that they aren't adaptable enough. With AGI, that changes. It will take a bit longer due to the capital requirements and factory build outs that would be needed.
GPT-4 is a very capable systems architect and can also implement the code. There are a few tools available to put it in a debug loop. Writing documents is a walk in the park for GPT-4. Emails or Discord chats or even perfectly realistic voice conversations are completely doable (I have that on my website).
At this point it's about connecting things together and looping them properly to automate a very high portion of jobs.
I think the answer is not employment but rather production. Think of something you can leverage these AIs that would be interesting or useful to someone else or some business.
Beyond that things like UBI and generally better integration of technology into government is going to be critical for our survival. Especially decentralized technologies and real-world resource data.
I’ve worked with tons of programmers like you describe. I’ve continued to tell them that simple UIs and CRUd interfaces to dbs are solved problems we should not be fighting with.
Maybe we shouldn't be, but it's still a problem that regularly needs to be solved.
In any case, I need to refute your argument, in my work as software engineer spanning more than a decade, I have noticed zero deprecation of my skills (Java, SQL, HTML/JS/CSS) (while keeping them up to date!) until now and only had to learn a few new complementing skills (cloud, docker, SPA, Kubernetes). The only skill that got replaced might have been "Java application server management" since that got replaced by whatever docker runtime is en vogue at the moment. I have worked for the government and met PL1/Cobol mainframe programmers that refused to learn Java and still got paid generously for their long term expertise.
I can see how you might think that... until you start actually talking in depth with enough actual users and executives and trying to get them to agree on how all that stuff should work and what it should be capable of.
Most of the development process is about trying to wrangle abstract ideas about how business logic should be implemented/improved from flawed humans who aren't great at communicating those ideas. Your 'simple' CRUD app still often has to be highly customized by someone willing to do the difficult work of dealing with people. And that's before you even start getting into working with more regulated businesses.
Code monkeys/plumbers using 'outdated' tech who can deliver something that makes a workplace more efficient in the long run will continue to be in demand. There was enough functionality in software by the 1970s to handle the vast majority of business needs. Someone still has to understand those business needs (which ultimately have little to nothing to do with software) well enough to translate them into something that works. Whether it works for those who are using it is all that really matters.
Yikes. Productive work is not just a way to earn a living but also a way to achieve personal fulfillment and happiness. It's a means of creating value and contributing to society. A person who works just for the salary and does not find any meaning in his work is not living up to his full potential.
If I could make money doing something I found a lot of meaning in I'd be doing that instead. Thing is, we usually don't have that option.
How many jobs in modern society are complete bullshit? A good deal of them, I would say. Why should people measure their happiness and self worth from these?
I wholeheartedly agree! I do know a few people that love their jobs and I envy them to no end, they are inspiring, shining suns. But I remain firm on my opinion that this is far out of reach for most people.
Capitalism maximizes profits, not happiness. The market for software development jobs is much bigger for people who know popular frameworks and are content with validating forms, querying databases, aligning buttons, sending reports, etc. It's a lot easier (and rewarding) to find fulfillment elsewhere.
For all but a select few this is an unrealistic fairy tail. Most of us just want to make money to better enjoy our lives. We were given or acquired certain skills to make money, out of juvenile interests or opportunities we used. That doesn't mean we enjoy using those skills. It would be very hard to find any other job without taking a massive pay cut, investing huge amounts of money, time and effort only to have a high chance you won't like your new job as well.
I see no job or career I am interested in: I hate everything the moment it becomes work. And I am no unique snow flake. I am part of the majority with that.
https://www.wellable.co/blog/employee-engagement-statistics-....
Things I enjoy don't pay enough to live a comfortable life. Tech does. So I do well enough at my job to pay for the things I enjoy, and hope I find enough edge cases at work to avoid burnout.
In a true post-scarcity society, where everyone has the freedom to choose a career based purely on fulfillment, your argument is excellent. Until then, however, it's not.
The only problem is that we live in a system that directs the gains upward and any costs downwards, and in so doing creates perverse incentives against people welcoming their redundancy.
Like sibling commenters, I love the idea of building something new with greater leverage. On an individual level, I'm looking forward to leveling up and finding new ways to be effective in my work.
Unlike sibling commenters, I don't think that should be our only option in life. It saddens me greatly that, given a new option to increase the effective output of a unit of time, we repeatedly choose as a society to profit monetarily (and with vast disparity in who benefits) rather than to give people more options in life than drilling on their jobs.
The industrial revolution promised people lives of relative leisure by replacing the need for much physical labor, but instead we concentrated the benefit to the few—and we keep making that same choice over and over.
But even then, self-proclaimed seniors are too scared to start their own startup(s) now because of (1) Unfavourable market conditions (2) VCs hesitant to raise money (3) ChatGPT will extinguish their startup; even if it uses "AI".
I guess this was the result of a decades long quantitive easing, near zero interest rate bubble of cheap money that had to collapse.
For example, I've been predicting the financial demise of Facebook for over a decade now (amongst other things for being too greedy), but Zuck's still doing well enough. Even if Meta is declining now, it still might have been rational for him to be so greedy over the past decade.
Date Weakly General AI is Publicly Known https://www.metaculus.com/questions/3479/date-weakly-general...
Date of Artificial General Intelligence https://www.metaculus.com/questions/5121/date-of-artificial-...
The latter includes this criterion: "Able to get top-1 strict accuracy of at least 90.0% on interview-level problems found in the APPS benchmark introduced by Dan Hendrycks, Steven Basart et al."
The APPS benchmark: https://arxiv.org/abs/2105.09938
Note that the predicted date of "stronger" AGI has moved quite a lot since GPT-4 is revealed, from late 2030s to 2033 at this moment.
> By the late 2020s, it's entirely possible that a "weaker" AGI will emerge.
What I didn’t say but should be a given is that these AI tools won’t be able to completely eliminate the need for any human touch, it will just reduce the need to the point where there won’t be the current huge demand for developers, and thus, the only people that travel down that path will be the ones that are truly passionate about development.
The question then is how exactly does one become better than AI at making software if no one is going to pay you to make software until you are better than AI?
The same issue with art - we need a sea of mediocre artists and a market for their work for great artists to emerge. AI takes over mediocre art market - all commercially driven art disappears.
You mean the same YouTube that routinely ruins people's livelihoods when it closes their accounts with no recourse? Because I'm totally looking forward to the day when that happens to my development tools.
"We detected that you are using our code to kill vulnerable children (aka orphans). This is against our TOS and we have permanently disabled your account. If you believe this was in error please log into your account and talk to our ChatGPT-powered tech support".
Worst case scenario is that it gets SO good at writing code that software engineering teams are severely downsized or are made obsolete altogether, and I find myself out of a job. I’m not expecting UBI to start falling out of the sky any time soon, especially while there are still manual labor jobs that robots can’t do.
Alternative scenario is that individual developers get somewhere around a 2x-5x productivity increase, but why would I want that? That doesn’t give me more free time - that just means I’ll be expected to do more work. Non-technical management already expects ridiculous delivery timelines; now I’ll have to deal with them asking “why can’t you have the whole project done by tomorrow? Why can’t you just have the robot do it?”
It’s a lose-lose situation and none of us asked for this.
It'll be interesting to see what happens when AI truly surpasses human level intelligence, as in, being able to completely replace human jobs, but we're not there yet. It's likely that when we reach that stage, the world will change dramatically and we will either live lives of abundance and leisure or face extinction :)
Third option, the workers no longer control the means of production, and we see levels of inequality that make the railroad barons look like they were middle class.
The bottom line is, at some point in time, automation is going to reduce the amount of human work which needs to be done, and render some folks unemployable --- how does society cope with that? Universal Basic Income is the only reasonable suggestion I've yet seen, but doesn't address the age-old problem of socialism --- it only works until one runs out of other people's money.
Back when computers were first announced, taxing CPUs so as to cover benefits for newly unemployed folks was suggested --- can we put that back on the table?
For a fictional take on this see:
Jevons paradox [0] proposes that as automation reduces the cost of labor then people will find new uses for automation, and this seems to be the historical trajectory. Hundreds of years since the industrial revolution and we still haven't run out of work to do (this could be better or worse given your philosophical premises).
> and render some folks unemployable
If automation truly causes more actually productive work to be done, then as a first-order effect there should be a surplus available to support these people without making anyone else (much) worse off. However as you observe the higher-order consequences of this are very much an open issue.
the current climate crisis suggests that we are running out:
https://dothemath.ucsd.edu/2012/04/economist-meets-physicist...
But we are there. This is a reality we live in for a lot of people. That's why the existential crisis in the OP.
If you're writing react/python/angular or something popular it seems to do amazing things and spit out entire websites (per demos).
Unfortunately, when I try to put together C++, Rust, or even C# using recent libraries like Blazor it chokes up. I fully understand at least one reason why (libraries and language features not being in the training data from 2021) but that makes me feel that perhaps software engineering at the cutting edge or niche is safe and still requires human reasoning. Not to mention things like properly understanding when and why to use certain data structures, real-world impact of coding choices, pricing, esoteric speed/efficiency improvements, etc.
I think there's still a broad general area where good, great, and amazing+ developers can operate without much threat and in fact using their knowledge and experience to leverage GPT-4 (or others) as a force multiplier.
With 32k tokens coming that's like 90kb total chars which 80kb could library or API docs.
Also it can easily be connected to things like pip or GitHub or Google to check documentation. And many tools are coming over the next few months that will put it in a debugging loop.
So maybe it's "safe" in the very near term but that issue of out of date training in no way prevents it from taking software engineering jobs.
I am working hard to build an AI system that can replace me before someone else does.
>...and I find myself out of a job.
Tell me you are the problem in the industry without telling me you are the problem in the industry.
Think about the hunter gatherer who was given a bow, but stuck with throwing rocks because he didn't want to get too efficient.
But more relevantly the instant bows were invented you're quota of mammoths to kill a day didn't go up to that maximum possible number + 1 (because sales guys). It stayed at 1 per week or whatever. It's not efficiency, it's management's unrealistics expectations of productive output that I hear being complained about.
Expect with more efficient hunting methods you could kill more than you did before per day, which meant less hunting days, which meant more time with the wife which in turn meant bigger tribe, which in turn meant you actually had to increase your quota.
Just because you are more efficient doesnt mean your manager becomes an idiot and starts to demand unreasonable output (and if you have an idiotic manager already then you already have the problem).
I have no fucking clue what you are even arguing for.
More, more, more, always more, the ideology of infinite growth belongs to a cancer. Quite literally a self-defeating, pathological mindset. More brought us here, to the present day, and it will probably bring us over a Seneca cliff too.
That's the thing about ambition. It means you will never be content.
As for the latter...I'd say GPT has increased my productivity and therefore allowed me to focus on the more interesting aspects of my work, rather than writing annoying boilerplate code and doing boring tasks where I don't learn anything. I almost never write my own boilerplate anymore.
More productivity doesn't necesarily mean more work. It does mean more focus on interesting work.
And so-called "senior engineer" salaries will now be brought down and deflated since they were inflated and unjustifiably high in the first place and are the main reason why these tech startups run themselves into the ground with little to no path to profitability.
I guarantee you that so far, the only winner in this is OpenAI. Not the 'senior engineers' building on top of someone else's AI API.
In fact, why hire 3 over-priced seniors when one junior with ChatGPT is significantly much cheaper? I quite find it funny that somehow, all hope is instantly lost because of a "AI" spitting out code will replace them. It just shows that the majority of these tech startups were just good at losing money and being solely dependent on VC cash.
Is this really true? I may be missing something (I probably am), but I didn't find much use for AI tools in my itsec/programming work. It's a nice tool to have, but I don't write that much boilerplate. I've tried to use it as a better Google, but it kept replying with made up nonsense (things I have problem with are usually niche technologies OpenAI is not good at - I expect it will get better in the future). So I find it dubious it will "10x my productivity" in the "upcoming years". Decades, maybe.
But maybe the future really is now, and I'm just being an old-timer who can't adapt.
If you want to do anything new or - god forbid - know of a better way to do things than what 90% of the population is doing (htmx?). Good luck.
So the future is anyone with that model access (the 8k tokens could have 20kb of docs which is still useful) who wants to really try.
Millions of creators grinding for pennies while the lucky ones that got in early and made it rake in the profits.
I think success in tech is going to become extremely pyramidal in the coming years. This is a huge shame, as this was one of the only fields out there where you could make a really good living without going to the "right" school for years and years and years.
10 years from now we might have the equivalent of what today costs 10 million dollars today. Automated farming means what today we consider high end and expensive produce becomes almost free. Automated transportation means that food gets delivered to you for almost nothing. Imagine you had a 95% off coupon on Uber Eats. Does that sound terrible? If so why? Because it also means that Jeff bezos gets a 2000 foot yacht?
Edit:---------
I'm getting a lot of doom and gloom respones. And you all are right, there are a lot of people who do not have food/shelter/cheap colleges. But what you all probably are not aware of is that 100 million people have risen out of poverty in India over the past 15 years. Your word view is being warped by the doom and gloom media. I would suggest reading just the beginning of the book factfulness. It will totally change your view of the world and probably make you much happier.
Most people also don't have $1000 for an emergency, live hand to mouth, and are dead scared of the cost and impact of a potential health issue. They are also overworked, underpaid, and with raising expenses, and sick of it, with depression levels skyrocketing. Having "a 10 million dollar supercomputer in their pocket" is not that comforting compared to that.
We've killed old style job security, cheap college education, affordable housing, the middle class and decent working class jobs, public infrastructure, and many other things (not to mention the environment), but in return we can have a rectangular gadget to access "all of the world's information in an instant" (which practically is just used to distract ourselves to death). Hurray!
https://www.marketwatch.com/story/more-americans-are-using-b...
https://www.cnbc.com/2022/01/19/56percent-of-americans-cant-...
This has been debunked many times. The source is misleading to the point of being deceptive, it is pushing a narrative. Per the US government, the median household has $1000 per month leftover after all ordinary expenses. A very detailed breakdown of this for each income decile is available from the BLS.
You can’t square “most Americans can’t afford a $1000 emergency expense” with “median Americans can afford to light $1000 on fire each month without impacting their standard of living”.
And it would be news to million struggling to pay the bills and rent (and not because they buy expensive lattes or new iPhones) that then can "afford to light $1000 every month without impacting their standard of living".
Especially for people whose standard of living is already working their ass off, perhaps in two jobs, and still scrapping to make it and not even thinking of affording to sent their kids to school, or can't even dream of ever being able to survive a need to stay off work for a month for medical reasons...
We have an abundance of inessentials. Housing is still scarce and food is volatile. Health care and education are expensive. Many people are sleeping on the streets or falling into lifestyles of despair.
Progress has been applied unevenly and most critically not to the factors of life that form the base of Maslow’s hierarchy.
Lots of properties being kept vacant so as to drive up rents/prop up property values, and it's difficult to get low-income housing built because of NIMBY.
We are going through 2.5 earth's worth of non-renewable resources each year in order to maintain our current lifestyles --- this simply isn't sustainable.
Let's turn things around:
- under what circumstances should a person be allowed to use more than 1/7 billionth of the solar energy which the earth receives each day?
- under what circumstances is at acceptable for a person to create more heat than 1/7 billionth of what the planet is able to radiate out into space on a daily basis?
IF we make it through...
Obviously the target number of starving children is zero. But the reality is that we are not going to get that number to zero overnight. It takes incremental steps.
But you’re right, the more level the playing field, the greater the competition.
In my niche(s), I still see new Youtubers pop up all the time that gain large followings and turn Youtube into a full-time job. Sure, they don't all become rich, but many have started earning enough to drive Teslas, so it's definitely not pennies.
Software development, as an employment opportunity, does not have these same dynamics.
This 10x productivity absense of a 10x expansion of programming industry (which is very unlikely) translates to less developers in general, including senior ones. Even more so in an economy like this...
>It means competition between companies will increase but it isn’t necessarily bad for existing software engineers, especially solo founders.
"Solo founders" is what? 1/10,000 of working programmers? And they're absolutely not the ones people worry about regarding GPT replacements...
I think I disagree. If software then becomes 10x cheaper, a lot of use cases that used to be too expensive to build now becomes affordable. At my own job, I think we could easily do 10x the business, because our customers need tons of tooling (for example for energy transition) but we don't have the people (among other problems).
The information society is a machine maximizing information exchange, something that only incidentally implies increasing profit or increasing productivity.
> I don’t get the overall doom and gloom towards LLMs on the software field.
From the second line of your comment:
> Now you don’t need to hire junior devs
Do you need GPT to put the two together? I think it's pretty obvious why folks are freaking out.
Every entrepreneur that ever existed.
> If you are a software engineer, this will output your productivity ten fold on the upcoming years. Now you don’t need to hire junior devs and can just build the product of your dreams with very limited capital.
And if you're a junior software engineer? Fuck you and be unemployed.*
Do you get it now?
* Until you can climb up the ladder where each rung is now 20 feet apart.
Are you seriously asking that question? What are the barriers to a junior dev writing the Linux kernel from scratch by themselves? What are the barriers from climbing from the bottom to the top of a ladder where the rungs are 20 feet apart?
Sure, start at the top, then it's great. Very few start at the top.
If I'm missing one, or a class of product with different barriers, I genuinely would like you to point that out.
Seriously, think about it a bit, without being sanguine.
The junior dev is inexperienced, in everything, and now has no path to build up that experience. No one's going to want their 18/22 year old amateur-hour "chatgpt make me a cloud app" (which is in competition against millions of others). So unless they're extremely lucky, they goto fail.
Maybe after 10 years of those failures they could build up enough experience through trial-and-error to maybe see a little success with a "chatgpt make me a cloud app," but how are they going to feed themselves the meantime? Maybe that will work if they have rich parents, but otherwise they're probably going to have to use up their energy to scrape by. So another goto fail.
This hypothetical scenario is literally like "pull up the ladder behind you", as all this experience and connections is something that a senior person has gotten while being handsomely paid for their time, but a future junior person may have to get on their own time and dime.
Ideas are a dime a dozen, execution is everything, and there's no reason to assume that random unemployed inexperienced people will be superior at execution.
If you are a CTO, this will output your productivity ten fold on the upcoming years. Now you don’t need to hire managers and can just build the product of your dreams with very limited capital.
If you are a VC, this will output your productivity ten fold on the upcoming years. Now you don’t need to hire anyone and can just build the product of your dreams with very limited capital.
Agree it'll definitely be amazing for creatives and solo founders, but how many ideas are really out there to be had compared to the reduction in workforce?
I don't know. But I don't see why you might not be able to ask GPT-6 or GPT-7 to enumerate (and patent and implement) all of them for you. Why do you think "founders" or "creatives" are special?
In the end, something like that is "amazing" only for the person who owns the most GPUs or manages to figure out the first effective meta-prompt.
No white collar job will be valued the same since GPT will basically be doing most of the work and we will maybe review it and steer it. We will just keep feeding it and it will know everything at the cutting edge of all fields.
the instructions for configuring google auth were off. I tried a number of different ways to get gpt to give me the right instructions, but to no avail.
so it was back to the old way, of spending a few hours reading google's documentation (which I'm doing today) to figure it out.
once I'm there, I feel confident I could better coach chatgpt to instruct me. though I wouldn't necessarily need the help at that point.
on the code side, staring at the google auth api code it had generated, I was faced with a hard truth. I didn't understand this code. to iterate with it, essentially to develop it, I would continue to be dependent on GPT. Even if there was a one liner needed, I wouldn't be able to come up with it on my own. I'd always have to rely on this outside "brain". How can that be more efficient than a tight REPL loop conducted by me, an evolving master of this API?
And how will we humans even maintain knowledge of these API surfaces if we are not putting in our hours and hours of repetitive usage of them? We become ignorant of the evolving capabilities of the computing platform. And chatgpt becomes useless without humans who understand what's out there, what's needed.
Spot on. It's a good time for existential reflection: Who would you have been hundreds or thousands of years ago? Who will you be now that technology is radically changing again?
There will always be interesting, creative challenges like programming, whatever form they take.
I'll just use this opportunity to recommend the video game "Ancestors: The Humankind Odyssey". It's a game where you start as an early hominid and have to gradually discover how to make and use rudimentary tools in order to take control of your environment, literally evolving in the process. It's weird and unforgiving, and it made me really think.
Given how I grew up ingesting science and science fiction alike, literally attributing half my personality to Star Trek: The Next Generation being on TV during my formative years? It's really hard to tell. I have very little connection to things which were possible before late 19th / early 20th century.
In my mind, being thrown back centuries in time, I'd spend my life trying to use everything I remember from present day to give everyone a head start on science and technology. Being thrown back centuries in time, but without the memory of specific things I've learned in present day? That sounds like a particularly sadistic death sentence.
Obviously you won’t be able to tell for sure, but I’d guess that 1000 years ago I’d probably be a serf, and 100 years ago, likely would have fought in a large war and likely doing some form of physical labor or subsistence farming afterwards, based on what my family was doing then.
In this light, sure, the me from 1000 years ago would most likely be a serf, die from malnutrition, war or robbery. Me from 100 years ago would probably be lying dead in the trenches of Verdun, or shot on the streets of Kraków, or otherwise dead in WW1; for military-aged males in Europe, I guess whether or not one got drawn into fighting was a coin flip.
Do you invest in a college education is that field is obliterated by the time you get out.
What about your debts if you lose your job and companies aren't hiring because they can just use AI for a 10th the cost in 6 months.
There's so much of talk about what these models can generate, which is cool in relation to plugins, but there's still a lot of interesting code to write, companies to build, and ideas to formulate, that an LLM cannot do on its own. If you're terrified of your software engineering job becoming at risk, I urge you to just take a beat.
There was a paper by someone @ Microsoft who tried to train a boardgame playing AI like this. The "best" models started losing to beginner level players from some point onwards.
I'm processing this news in realtime like many of you and forming a plan:
1. Understand how LLMs work. I've heard the Wolfram paper is good; open to more suggestions here.
2. Continue to practice using real implementations of LLMs including ChatGPT and co-pilot.
3. Finding painpoints within our company that AI can make more efficient and implementing solutions.
If anyone feels the same way and wants to form a working group with me, give me a shout. Email is in my bio.
For the understanding part, Andrej Karpathy has a YouTube playlist that explains neural networks. I made a start on it today and found it quite accessible.
https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...
Basically we need to equate "safety" in LLMs to mean "being open-source".
OpenAI keeps talking about "safety" as the most important goal. If we define it to mean "open-source" then they will be pushed into a corner.
I think open source is a reasonable component to safety, but I wouldn't want to make them equal. Open source may be necessary for safety, but I wouldn't call it sufficient.
For example, assume the source code, the model, the training data, and all the model weights are open source. How do you know that the model was actually trained using that training data? Very few organizations have the capacity to train models at this scale themselves.
Another way to put it is to make it more accessible to everyone, right?
The opposite of that is happening to nuclear power. They're actually trying to stop any more countries to have the technology at their disposal. So no, make it "open source" doesn't make it safe by any stretch of imagination.
reactor blueprints have been accessible to IAEA members for something like 50 years
Whenever I see this I simply think "monopoly". It smells of anti-competitiveness and is a kind of open forum lobbying to restrict who gets to lead the AI wave (and make a shit tonne of money in the process).
The part I'm finding is kind of a shock to me is the impact of the centralization on what you can even think about doing. If your application falls under their random definition of "unsafe", then you can't do it. Not even manually, probably, because the infrastructure for that will go away. If your one off question or task doesn't meet their approval, it doesn't happen.
Basically not only do the owners of these things become the only really important people in the economy, but they also get a new kind of direct control over people's lives.
Because yeah it works fine for basic programming things but I believe you need to know wtf you’re doing when it comes to anything more complex, even something basic like some of our single endpoint services.
I suspect many large IT organizations are like this.
Wasn't this the final objective of the programming languages abstraction evolution? From Binary/Assembly to Natural Language Programming? I think it is awesome that more people will be able to create software/products as this accelerates innovation cycles a lot.
And, for now, I believe devs that don't rely solely on copy/paste coding from stack exchange don't need to worry about their job stability no?
Others have said exactly what I'm thinking, welcome to the age of the micro-startup, 1-3 engineers, designers, product mangers building some very cool, albeit niche products.
"Domain expertise and insight into a potential market" won't get you a working product that you can sell.
I envy the people who are bottlenecked on their typing speed and benefit 10 times more from the chat bot than I do.
Until someone starts testing this and finds a bug. And then AI will say, hey, there is no bug, I don't make mistakes. So you need a human to look on the code, a huge pile of spaghetti code with cryptic names and conventions, code patterns that fell out of fashion years ago but, since there is a lot of code that uses them, AI thinks they are ok.
How long it will take to fix anything, how long it will take to extend the code?
The code it generates is by no measure "a huge pile of spaghetti code with cryptic names and conventions".
I was sceptical myself before trying GPT4. I asked it to change the Python C internals for a new feature, and googled to ensure the description doesn't exist anywhere. It came up with very good changes and explanations.
And this is all not even mentioning the pace of improvements. It didn't take too long to go from GPT3 to GPT4. Even if the pace slows down, it is still huge.
> it does not need good variable names or functions/methods/class names,
It's the exact opposite. It's too good at naming things. It insists to use variable/function names that make sense in plain english, and often make mistakes when the API has inconsistent naming, or consistent but unusual naming.
For example, it makes mistakes when writing code that use "Loop" in Blender API. And the reason is quite obvious to me: because Blender's "Loop" is not what loop means in plain english.
Writing is definitely on the wall for outsourcing and MVP-style work. GPT can create a landing page and a backend/frontend for a business _literally today_. You just have to ship it, but it won't be long until that isn't needed.
There will still be a lot of value in understanding how systems work and interact with each other, at least until ML is able to build and maintain entire systems.
Until that happens, there will still be a lot of value in being able to dive into codebases and refactor/optimize as needed, at least in the medium-term.
Once platform engineering is mostly automated and running AI-generated binaries is de-risked, then code quality doesn't really matter. Hell, _code_ won't even matter at that point.
To me, this sounds a lot like "at least until ML is able to reach level 5 self driving". We don't even know if this is possible yet without AGI (which we also don't know is possible). We can get close, but... that last 1% is a bitch, and it makes all the difference.
Did OpenAI just commit a trillion dollar mistake?
I don't see this
I don’t think convert it in and out of proprietary standard is that difficult?
There is little to no vendor lock-in effect
What insight!
/s
Your original point was ridiculous, tone deaf, offensive and completely without substance other than to wave the victim flag about _something_ I guess? Who knows.
I missed this. Can someone show me what he is talking about?
I know it used a huge amount of energy / GPU cycles / time to train, but now that the weights are computed, what's involved in running it? I know the model is huge and can't be run on an ordinary developer's machine, but I believe requests to it can be batched, and so I don't really know what the amortized cost is. Right now, this is all hidden behind OpenAI and its credits; is it running at a loss right now? How sustainable is using GPT-4 and beyond, as a day-to-day part of professional life?
We still need actual experts to vet the code LLMs produce and to choose the optimal solutions. This is what senior devs have done so far with junior and mid level devs always. There are people who can write code, but someone needs to review and approve what they have done.
Obviously LLMs will also eat into that space, but before we come up with AGI LLMs alone won't be able to completely replace humans in software.
My first reaction was to be afraid for my money-making skills. My second reaction was fear about us ourselves making ourselves irrelevant--that fear still lingers.
My third wave of fright, cemented by days burning my eyes looking at a screen parsing logs and trying to figure out bugs for my corporate master, was, "when did my imagination go for a vacation? Old boy, don't tell me now that you have run out of ideas of things to make, of things to have an AI army to help you build." And now I dread that all of this AI is just hype, that it will never be good enough to come for our jobs without also coming for our jugulars, or that we will make it too damn expensive to matter[^1].
-------
[^1]: Capitalism has a way of leveraging economies of scale to make certain goods cheaper. But there are physical limits--what if Moore's law with regard to power consumption is really dead, and we as a collective really decide to spare power?
It's either my imagination that has gone for a vacation, or yours is running wild, but that is the one thing I really can't see at all. Reducing power consumption? I don't think that's happening any time soon, or ever really.
Some day it will be. Not those ones, those ones are only hype. Also whether or not they'll come for our jugulars depends on what they are commanded to do. But we will get them eventually, and they will be as good as articles like this pretend the hyped ones are.
The funny thing is that nobody will use the current panic to prepare. And everybody will use the current panic as an excuse to avoid preparing once the real AIs come. So they'll get us completely unprepared.
If the LLM has seen lots of instances of usage of an API, it can write code to target the API. It can generalize to some degree, but things go off track the further your requirements are away from the training data.
If your code is a lot of duct tape between well-documented, or at least well-named, APIs, that code can be automated. Which is great. That kind of code was always boring to write.
I'm less convinced that LLMs will be great at inventing new abstractions to map to a problem domain, and wiring up these new abstractions in a large codebase.
They'll need augmentation, fine-tuning, guidance, and it's not clear how well it'll all fit together, and where the limitations of the tech will show up as capability cliffs.
It's also a good time to really take our heads out of the sand and re-evaluate how we expect people to learn civil engineering if their only teacher is a minecraft world. You might get some people that are perfect in minecraft. The rest will be hopelessly stunted. Pretty soon it'll pivot to materials engineering to figure out how exactly a minecraft block adheres to a surface because we lost the original irl way to build a bridge.
When I need to ask for boilerplate code for fetching a web resource or using a well-defined API, ChatGPT is great.
ChatGPT has made the mundane plumbing a lot easier. It is a threat to plumbers at this point. Many of those plumbers are now freed up to do more valuable work. I am happy to have it, so I can focus on higher value work.
If your only skill is at this kind of low level plumbing, you are in danger. But I doubt this is the case for most.
Today.
What happens when it understands computational geometry and can calculate an optimal strategy to apply it to a dataset and end goal you provide?
(My intuition is that ChatGPT, like all technologies before it, will end up making more wealth and more jobs possible.)
Why is this extraodinary? What would be the advantage of going through all the effort of defining a new format just to create busywork for people trying to integrate with you?
It's not like there would be anything stopping Bard/Alpaca/etc. from reading the same format as OpenAI.
Key thing for adoption is to make models smaller and more context specific (to make them smaller), we've seen how LLaMA was downsized to run on commodity PCs, we've seen how Stable Diffusion can run on mobile phones. Even when we have to use larger models remotely, cost and ownership matters.
> Accepting the term "intelligence amplification" does not imply any attempt to increase native human intelligence. The term "intelligence amplification" seems applicable to our goal of augmenting the human intellect in that the entity to be produced will exhibit more of what can be called intelligence than an unaided human could; we will have amplified the intelligence of the human by organizing his intellectual capabilities into higher levels of synergistic structuring.
Now that the computers can talk and think and program themselves, and we can expect them to become exponentially better at it (to some limit, presumed greater-than-human), there is approximately only one problem left: how to select from the options the machines can generate for us.
It's still an open-ended challenge, it's just a new and different challenge from the ones faced by all previous generations. And again, just to repeat for emphasis: this is the only intellectual challenge left. All others are subsumed by it (because the machines can (soon) think better than we can.)
About twenty years ago, I had a professor explain to the class that Rational Rose would be replacing us all....yet here we still are.
Maybe it could just be an alternative syntax for an existing language which is more optimized for input/output to an LLM.
I’d guess that the languages with the fewest implicit behaviors (so no Scala or Haskell) would be easiest. Maybe Go is the generation language of choice?
> OpenAI made the extraordinary and IMO under-discussed decision to use an open API specification format, where every API provider hosts a text file on their website saying how to use their API. This means even this plugin ecosystem isn’t a walled garden that only the first mover controls. I don’t fully understand why they went this way, but I’m grateful they did.
So accelerant, definitely. Beyond that, I'm on the sceptical side but accept there's quite a chance that's the wrong way to bet.
In that endgame, anyone who can speak can command AI can do whatever they want it to do. Any kid with a louder mouth can outwish the wisest man on earth.
That means shortsighted impulsive criminals can use it to learn how to steal. Shady politicians can use it to astroturf entire campaigns. Everyone knows the tropes but it bears repeating as we all march dumbly towards what's coming.
It is far easier to destroy than it is to create. And humanity aside from China has not demonstrated any sort of sensible strategy to temper the tendency of destruction to outpace harmonious creation when it comes to AI. The more I see AI emerge and see people use it for exactly what people fear it shouldn't be used for, the more I feel China's centralized adoption of it, though maybe not "feel good", might be the DNA that survives in the natural selection of societies.
I know of one person who pays for GPT, and I'm guessing they use it to astroturf demand for their own business's products, since that's what they were doing by hand when they were younger.
It's a good point and some have already got this to work:
https://twitter.com/vaibhavk97/status/1639281937545150465
Given that there's no technical obstacles to drop-in compatibility here, I wonder if we'll soon start seeing exclusivity requirements and such.
It's one thing to ask GPT to write a high level script to trim 5s off a video using ffmpeg. It's another thing to ask GPT to make ffmpeg, or even to make a specific modification to ffmpeg.
It's hard to say how good GPT will be at real-world programming since we currently can't try it out. Maybe it can scale to the task, or maybe it can't, but i wouldn't say that programming is "finished".
OpenAI's plugins are equally temporary. Right now they will be generating actions through APIs, but GPT4 is probably already capable of performing the same actions on your browser. All it needs is a "control my browser" plugin that allows it to make that reservation on expedia, without expedia having any control in it. It will inevitably eat the world again
Doesn't this show that we can now use this technology to generate and execute code for modest problems that have already been solved, while we can spend more time on even more complex problems?
What chat app? Is this gpt-4? I haven't seen anything executing the code that is generated. So is the above quote a hypothetical or what?
As of right now, even if ChatGPT were to generate 99% accurate responses, it's quite a chore to communicate with it in full sentences. I don't want to have to explain my business in full painstaking detail and then upload tax documents to a system that can then output an answer in book form back to me.
Interesting! Somehow I missed this. https://spec.openapis.org/oas/latest.html
90% of programming is communicating with other people - chatgpt can't talk to people.
It also can connect to your Notion, Slack or whatever
Then I predict we'll get more business analysts than programmers, since managements will still need people to translate their needs to AI.
Why would analysts be harder to replace than devs?
The question is - how will competition influence the job market? if everyone has AI, everyone has the same powers. So how do you differentiate yourself? You put more humans in the loop, like "human plugins". You need humans to extract the most from AI.
or just build an embedding database the pulls the most semantically similar paragraphs and let it use that as a basis for the conversation.
The job market will still almost be the same, that capital and networks will net you businesses.
The problem is how to regulate duplication, because IMO with power of AI patents are basically almost useless.
Once we can send LLMs to meetings with each other, we can move down to 15 hours of purely joyful work :-D
Have we already solved AI safety problems? It seems like LLMs can now execute shell commands on our computers.
[0]: https://openai.com/blog/chatgpt-plugins#code-interpreter
That the draft happened to work on the video clip is more luck than something you want to bet your engineering life on.
You still need to go through an verify every character this statistical package spits out - it is not magic - it is just a probabilistic machine.
There will be no Post-GPT computing world, just the Turing police and console cowgirls.
Or the end of the beginning (of software development)...
What do I need them for if I can get equivalent code written for me on-demand?