AI and the Automation of Work
ben-evans.com
ben-evans.com
So I think that any job that can be done by a non-human is up for grabs, but for the sake of the argument, let's restrict it to manual jobs where being human doesn't matter. For example, a kitchen clerk at a burger joint doesn't need to be human, as long as it does all the tasks it needs to do and doesn't put anybody in danger. A security guard at an airport can be a machine gun with wheels, it is more intimidating that way. Picking up coffee cherries from a coffee tree can be done by a machine which is smart enough.
But assume that there are jobs "high in the ladder," for amazing engineers and creatives super-charged with AI powers. The problem is that not everybody can climb that ladder. And that's a problem we are already seeing. Right now, if you are highly qualified, you have no trouble getting a decent job. But if you are not, you are fighting for 10 USD/hour shifts somewhere. Imagine if 90% of those shifts are gone. Compound that with our social media (and AIs optimizing for engagement there in) making our kids a little dumber by stealing their attention for long periods of time.
It doesn't look good.
The low-hanging fruit IMO are most likely to be anything that is done entirely via/inside a computer. Because interacting in the digital world is much less of an issue than having to deal with digital + physical world like manual jobs do.
That's not to say it will take all computer jobs but those intuitively seem like they will be the easiest for computer-based AI to do.
Something like working in a kitchen is a long way off (or at least will require a redesign of kitchens from the ground up or humanoid robotics that are effective, reliable, and reasonably cheap). Whereas if your job is already entirely done via computer with little real world engagement, then it will be much easier for AI.
If you mean a robot that could clean in any kitchen without breaking anything, I'd say at least 20 years. As in you could move it from kitchen to kitchen with no install needs.
I suppose another way to look at it is this: how many businesses use robot cleaners to vacuum/mop their floors? Because as far as I can tell that is already solved (basically) but it isn't done because the cost/benefit/efficacy just doesn't add up.
What do you think?
There would also be a larger "boxes" available for fixing larger things. For example, your washing machine broke up, you put it into the "communal" box that is like 6x6 meters, and inside it, it will repair it. Or you could have a communal box that works like a kitchen (but IMHO the biggest problem with AI kitchen is that we cannot easily automate the human taste and smell).
I think we're pretty far away from a robot that can take an arbitrary problem and fix it accurately. I'll be excited when I can take my car to the mechanic and he's supervising a robot changing my oil.
" I think if we don't solve general intelligence first, we should be able to get there in three to four years"
Very unlikely. I mean you can build something that roughly does that already today, but it will break things and/or it will be so slow it is unusable. Also you have to take savety into account. The robot may not wash the cat or the kid and not accidently break them. But the level of required autonomy for the task requires a machine that can be dangerous. And the recent rise of LLMs really don't help with reliable doing tasks in an unfamiliar environment.
Making cars drive save everywhere is an equally hard target and despite all the investment, I don't see us getting there anywhere soon. And there is not the investment in making robot dishwashers, because we have common dishwashers, that do require manual labour, but they work. A humanoid robot on the other hand, that can put the dishes into the dishwasher, but requires some learning, this might be possible in a few years.
And of course, if we crack an AI that can work on AI better than humans can, that solves - or at any rate obviates - all other AI problems by proxy.
At least in the US, the parasitic labor unions are explicitly fighting such automation:
https://www.theatlantic.com/technology/archive/2019/01/autom...
https://www.huffpost.com/entry/american-workers-jobs-inequal...
Cleaning robots may not yet be cost-efficient everywhere, but they can't be far off.
The same reason we didn't see too widespread automation in industry. You actually need a lot of engineers to automate systems for their specific purpose. Today robots sold the most primitive parts but even there rework is needed. There are exceptions where scale and value of the good allows for it.
An AI currently isn't even capable of doing excel business analysis without the data being checked. This check has to be done by an analyst as well. It cannot even do book keeping without errors and the work to check for errors surpasses doing it yourself.
We aren't there where anyone has to fear about their jobs from AI compared to general technological advancement. If so I think the first impact will hit the click generators (good) and artists (bad). Although in both cases you need people to generate new concepts. True that you cannot get rid of cooks ever though.
> An AI currently isn't even capable of doing excel business analysis without the data being checked. This check has to be done by an analyst as well. It cannot even do book keeping without errors and the work to check for errors surpasses doing it yourself.
Here I think you are thinking about this wrong. AI is going to be built directly into Excel so I think it will harm a lot of especially entry level type work for analysts. If any part of your job is using microsoft/Mac software I think you will see a radical shake up in work (that will still enable more people to be entrepreneurs etc so on balance a good thing).
I'd go further and say:
Excel is the automation: "computer" used to be a profession rather than a machine, and a linear regression line-of-best fit is basically the same maths as a single (no backprop) perceptron, which we used to have to calculate by hand.
I think AI will probably eliminate almost all future knowledge worker jobs through simple attrition. In 20 years, perhaps 1 human knowledge worker will be doing the work of 100 (measured by todays productivity) augmented by integrated AI in the tools they use.
I am convinced that short of certain trades, we will watch AI erode everything. Personally, I am on a mission to convince my 2 year old grandson that becoming a plumber is the key to his future wealth.
Lump of labour fallacy?
I do see worker dislocation in certain sectors though, can't see how LLM's won't substantially shake-up office work especially as Microsoft is already busy building them into the Office Suite.
Which was my point. Knowledge workers (and the types of people who gravitate to knowledge work careers) will be dislocated and what work remains may not be as rewarding both financially and personally to these kind of people. People of course adapt, but would that person sitting on the autism spectrum who can thrive and make exceptional income as a SWE be able to adapt to a less cerebral career? Will there be mentally rewarding work for those displaced? Those are real concerns and a possible butterfly effect of AI.
My hope for my grandchildren is to pursue careers that can weather the automation and AI storm. Becoming a plumber might be literally shit work, but it’s one trade that is AI and automation proof for many years to come.
AI will become a tool instead of it replacing SWEs. Chances are that software engineer even become more essential through that if AI is integrated in every system.
Once the bulk of the coding can be offset by AI, you probably just need elite SWEs as the check to the system. The lower quality SWEs that fill that extra pair of hands role that we need now will go away because the code they produce will be of inferior quality to the code that AI will produce. They will be an anti-productive variable to the equation and will be gradually lost to attrition.
for instance, transcription?
Factories that make food are already a thing.
I didn't know there was an entire TV series dedicated to just this topic until 5 minutes ago, but I have seen at least one short segment about the topic, specifically Viennetta ice cream: https://en.wikipedia.org/wiki/Food_Factory
three simple words that are key here. as long as nothing changes, the algorithms that look to the past to predict the future will remain accurate. as long as.
[0] https://twitter.com/alexblechman/status/1457842724128833538
The war on terror really drove people crazy...
The "answers" the US came up with rather exacerbated the problem. Most also don't work with homegrown terrorism at all.
Using AI to profile such potential endemic perpetrators is ironically just another instance of trying to fit the problem to the tool.
Now you mention it, despite the two respective national stereotypes, the only airports where I can remember seeing machine guns on the security team were one of the London airports.
If I saw machine guns in SFO or JFK, I was too tired from the flight to remember.
The dispatch time for armed police has gone up because they are doing a regular police job, while armed, rather than waiting for the call up.
Don't remember whether it was Incheon or Gimpo.
What I see now is occasional police officers with their standard sidearms, but rarely see anyone more armed than that.
Whereas in the US, every cop is armed but there's more checks on the more strict stuff.
Your airport police want to be armed but there's no need for them to be heavily armed. So in the UK they all are, because it's the same people. In the US the heavily armed folk are deployed elsewhere (SWATing someone's Twitch stream if the media tells me the truth...)
Have you been at the Charles de Gaulle Airport in France? The security people there look like paratroopers[^1]. It's terrifying, and it's a reason I avoid that particular airport every time I fly. It sends the message "Paris is an unsafe place to be, we are very paranoid, blame the others, don't come here." The machine guns have no business among civilians, being with robots or people, period.
>> That is a job which is 99.99% about understanding human intentions in confusing and ambiguous situations, and where making a mistake will literally kill people.
Yes, if you ever are allowed to pull the trigger. And in that case, I wouldn't trust machine nor person; I have been profiled on the basis of my physical appearance more times that I can count.
But in some cases, like the airport above, the goal and effect is to simply terrify civilians. You can't deny that a machine would do a better job at that[^2].
[^1]: Just search Google images for "Security at Charles de Gaulle airport." I've been there, and that's exactly how they look.
[^2]: Here, have a Dalek with a white paint job: https://www.knightscope.com/ .
In europe there's 1-2 orders of less police brutality and issues overall, but the tradeoff is you randomly have military police walking around with rifles.
They're quite friendly actually, I've ended up getting directions from them once or twice when trying to catch a connection.
Also the FAMAS is a beautiful piece of French Engineering
What does suck about CDG is how horrifically unorganized they are. Like "print your boarding pass, but the printer was out of ink, so the barcode didn't generate, and they just *wave you through* the check with an invalid boarding pass because it happens all the time
that is what scares me, not European cops with guns
I doubt any relation of these two facts, or cannot see it.
Protest is part of French tradition
You're a braver person than I. I cannot imagine a circumstance where I'd be willing to even approach officers armed with machine guns, let alone ask them for anything. I'd just be itching to get out of that place ASAP.
Friendly-fire military incidents are common; while it's associated with inexperienced troops, the reality is that experienced and "elite" troops still have many friendly-fire incidents. And then there are all the civilians that are killed as collateral damage.
Of those that do, firearms officers - why would they carry a handgun?
And you are right - there is of course an element of security theatre!
To be honest, if we're talking about most minimum wage jobs being performed better by AI (I would imagine being armed security, at least in the UK is a lot more than min wage, because of the responsibility and trust of course), I'd worry less about their AI machine gun at airports vs the existential questions around there being a working class (in the original sense).
Trains and planes already operate completely autonomously 99.9% of the time. Yet we still have pilots and train engineers there to handle the edge cases because having one or two people to keep an eye on a million dollar machine that can kill hundreds of people and cause billions in property damages if handled incorrectly works out to be a pretty good deal.
This is the main problem with self driving cars. They can kill people.
This is the biggest misconception about AI and automation: manual jobs will be the last to be automated, because making robots (which is specialized hardware) is much, much more expensive than making software. That's why our 70s dreams about robots haven't materialized, despite Japan going all in on robots, and why Besos aims to “algorithmically control workers' body”.
> It doesn't look good.
Having machines do work isn't a problem in itself, in fact that's what the industrial revolution was about, the problem is power and whealth grab by a minority of people owning the means of productions…
Jesus. Fucking. H. Christ.
RoboCop (1987) should be required watching before anyone is allowed to touch AI.
Out of all the dumb things people could suggest AI for this is without a doubt the dumbest.
IMHO such jobs won’t exist. Once AI have reach AGI level there is nothing that a human can do that the AI can’t. Might as well just add more AI.
Frankly, what scares me is everyone rushing to advance AI but no one seem to be putting any effort into figuring out what to do about society once humans can no longer trade labor for money.
Folks pay for handcrafted items today, even when there are cheaper and better mass-manufactured items available.
No matter how advanced AI is, humans will pay for things to be done by other humans.
Sam Altman himself spent quite some time on the idea of basic income (2016)
https://www.businessinsider.com/inside-y-combinators-basic-i...
I think the last thing airports want is for their passengers to be more stressed than they already are.
What would be the use of an AI picking coffee beans if there is no one with a job to buy the coffee beans? There is an unresolved issue here for sure. Unless the machines are self-sustaining, there's not going to be much use for multi-billion dollar data-centers and millions of dollars in electricity bills and expensive robots if there is absolutely zero need for anyone to be using these systems.
Tesla has robots making cars, ok, who has a job therefore money to buy a car? Many, many of those who are out of work from the Tesla factory will no longer participate in the economy, and that Robot cop at the airport? That will be pretty lonely once people stop flying for their profession, which would be a large customer base for air travel.
A shock this big to the economy could happen quite quick and it would be fairly devastating.
I love to think about all this and play with thought experiments, we have very litte clue where this is going. I actually think Robots are more of a human problem than we realize. AI systems probably have, much, much less need for robots than biological species because we actually heavily depend on manipulating the environment for survival.
The IMO dystopian move is to just let the AI employ humans as 'hands' for subsistence.
Software testing is easy to fake, which means it’s particularly easy to claim automation accomplished.
Therefore, all these people thinking they are going to automate what I do when I test? No. You have no idea what I do. But what you can do it convince ignorant managers to buy fake software testing. That’s the real danger.
Imagine if a company came around claiming to have revolutionized blood testing with radical tech, labeling its competitors slow and obsolete. The sober and responsible players in that industry may experience real damage before people figure out that Theranos is a fraud.
I think it’s a good idea to keep an open mind and see what happens. You might be right, but if you’re wrong, you’d be better served to be ready for it.
Which seems more likely: that it’s impossible to automate testing with AI, or that it might be possible?
They can't make the same argument because the experience is first hand and can't be faked.
Developers generally hate writing tests, so I suspect more and more unit tests will be generated. I've worked with someone who does this and it seems insane to me, although I never got to see the generated tests they were referring to, so I can't give specifics.
IMHO the gap is that human-written test understands the (unwritten) specification (which might be as simple as "program doesn't crash"), and tests the program against it.
I am of the (pretty radical, it seems) opinion that the (especially unit) tests that just take the code as the specification, instead of having specification coming from elsewhere to test against, are useless. So you cannot build a useful test just out of the code, because you always need to test an implementation against a specification, or, two implementations against each other.
In that case, if I have code that satisfies that specification as demonstrated by say acceptance testing or years of use, I will happily opt for test that take the code as specification. I am not using those tests to prove the code "correct" but to prevent regressions during refactor or new feature development later on. Worse case scenario they should keep the program as broken as it is now or has been for years.
Then you don't really need to have the tests written, what you need to do is to compare the output and behavior of the two code bases before and after refactoring. I mean why use a low fidelity approximation of the original (i.e. test that only works in some cases), if you already have the original at hand? (After all, how can you predict ahead of time, which features of the original version will need to be tested as preserved in the new version? Isn't writing regression tests in advance a form of premature optimization?)
From the view point of testing, you're comparing two implementations. So again there needs to be some understanding which one is the correct specification - the original one or the new one?
It would be more useful if the industry actually treated the two cases - i.e. testing against specification and regression testing of the refactored code - as completely separate, instead of trying to push unit tests for everything. Because in the latter case, ideally you can prove that the refactored code is doing identical thing, which is stronger than testing.
Increasing QA maturity level:
– B.Beizer – „Black-box testing“ ($), „SW testing techniques“ ($)
– ISTQB Foundation level
– RTCA DO-178C ($) / EUROCAE ED-12C ($), DO-248C
– Joint Software Systems Safety Engineering Handbook
Test analysis specification-based techniques (EP, BVA, EP DI, STT, TCl):
– ISTQB „Advanced Test Analyst“, „Advanced Technical Test Analyst“ courses and Syllabi
– BS 7925-2 „Standard for SW Component testing”, ISO/IEC/IEEE 29119-4:2015 „Test techniques” ($)
– IEEE 829:2008 „Standard for SW testing documentation” ($)
Test analysis failure/risk-based techniques (FMEA, FTA, RCA):
– C.Wilhelmsen, L.T.Ostrom – „Risk Assessment: Tools, Techniques, and Their Applications” ($)
– C.S.Carlson – „Effective FMEAs” ($)
– MIL-STD-1629A „Procedures for performing FMEA”
– NUREG-0492 „Fault Tree Handbook”
– NASA Fault Tree Handbook with Aerospace Applications
– DOE-HDBK-1208-2012 Volume 1 „Accident and Operational Safety Analysis Techniques”
– TOR-2014-02202 „RCA Best Practices Guide” ($)
I think the specification from elsewhere is all about the business value that the code delivers. At the end of the day, the business doesn't care about how exactly your code is decomposed into classes/functions/etc, the business cares about 'can I register a customer' and 'are customers prevented from writing to resources they only have read access to'. Of course there is some requirement for the code not to be a complete mess, as this has impacts delivering features and reducing bugs down the line, but with tests that sit closer to the value the code delivers than the structure of the code, you're free to refactor and the tests will tell you what you've inevitably broken. I find this considerably more valuable than unit tests.
The next problem is that ChatGPT is not thinking critically about risk. It thinks (well, no it doesn't think... let's say it "treats")... it treats testing as a process of demonstration. Demonstration is not testing. Demonstration proves that functionality is possible, but not that it is reliable. Thus, it produces checking code that is shallow.
The next problem is how are you going to explain to ChatGPT what your product does? Do you give it all your source code? Do you give it all your Jira tickets? The demos I have seen are toy examples. Nothing on a realistic scale.
Let's say you feed it a whole spec and source code, somehow. The next problem is omissions. ChatGPT arbitrarily stops producing output. You then have to carefully check its work to see what it has left out.
What you are asking ChatGPT to do, really, is to write code that will maximize the probability of spotting a real bug, while minimizing the probability of a false positive. But it will arbitrarily focus on only those kinds of bugs that it can easily discover. This is the oracle problem.
The next problem is that ChatGPT halucinates or misunderstands, so you have to correct its mistakes. Sometimes going through frustrating iterations of prompts, like a man trying to pull a mule through town.
You also have trouble with test data and data setup. Only in toy examples is this not a significant problem.
ChatGPT produces conventional results, and that does have some value. But it's not enough for professional work.
I just think the people who casually say that ChatGPT is going to easily handle these things are pretty stupid, under the definition of stupidity as "the refusal to think."
It's certainly a very interesting technology. But in it's current form, I just don't see it replacing most jobs. Will it replace jobs if we scale further? I don't know, but my intuition says no. I feel that we will need some other major breakthroughs in AI to get there.
Nice slogan... is this an ad?
Explain it to ChatGPT and it gets you 95% there.
It's a huge time saver. Sure you could do it by browsing the web and sticking together a bunch of pieces, rewrite it twice etc.
Already I didn’t realise my docker container was saving data to the NON-PERSISTENT data volume. But that was a greater reflection of Chatgpt taking me beyond my knowledge limits, and me being in a place where my ignorance of my ignorance can lead to not-great consequences.
My ChatGPT experience is not that different from the "good old days" of searching the internet. This time the search is in latent space which has up- and downsides.
But all these things were possible before ChatGPT.
If I was more intelligent I would be less impressed with it because I would probably already know what it has taught me. I understand the thermodynamics of that submarine implosion now but if I had taken any kind of physics I would have already known all that. I didn't take physics though. This pretty much extends to all subjects.
I will have to have it teach me to setup docker containers.
The exercise I really need to get on is to try make something in a language that I have no clue about. Like Rust or Go I know nothing other than they are programming languages. Then it is an exercise in the skill of going from zero to not zero with the help of AI and that not zero will hopefully keep scaling up and up over time.
I think it works because its not a very complex app and since it's gated with a sufficiently high entry, I don't expect a ton of traffic, hence scaling is not an issue. I also chose the easiest options to deploy (Vercel for the frontned, Heroku for the backend), which might end up costing me in the long run - but I figure I'll get that fixed later
My year in tech regrets ([insertyouwebsitehere].com) 7 points by spaceman_2020 on Jan 15, 2014 | past | 3 comments
I've tried phind and it's almost always worse than a Google search. It often has a reasonable answer, but with Google I typically find more complete information and sometimes better rebuttals and alternatives.
I view the thing as a neat toy. Can you learn from it? Maybe. But if you think this allows you to learn something you couldn't otherwise, I wonder if you're really being honest with yourself.
I have no delusions that this is making me a good coder, or that I can carry this beyond the prototype stage - I fully intend to find a CTO/co-founder. But for someone going from 0 to 1, this has been a fantastic tool.
While generative AI does make writing some things more efficient, it almost always has a bland, generic style that is immediately recognizable. The only jobs it will be replacing are ones that didn't require much effort in the first place.
Of course, that might impact the number of available jobs and it's something that need to be taken under control.
But, if talking about writers, while AI won't realistically replace imaginative authors and investigative journalists (at least, certainly not in the short term), it will likely replace (some) copy editors and dollar-per-thousand-words "journalists" that have to fill tabloids and free subway "newspapers". While it might be a financial loss for the people involved, the society will hardly sink due to that. In fact, if I feel optimist, in theory it might reduce costs for publishing houses and newspapers, to invest more into quality (human-made) content.
What I worry about is human greed in relation to AI, be it GPT or anything else. The incentives are lined up to produce something akin to AGI and the tools are becoming more tangible. The goal is to tap the honey pot of automating current knowledge labor first and foremost. Later, other areas of labor.
I can't see anything on Twitter anymore, but before that happened I think I hadn't seen many of the entertaining ChatGPT conversation screenshots anymore. That entertainment use case had the hallmarks of faddishness.
It seems far more useful as a productivity aid though, but as the OP discussed, I doubt the chat prompt UI is going to be the paradigm for that. Copilot is already mostly a more useful UI for me, and I suspect that's where this is headed, with a bunch of different specializations (and even more failed attempts).
I think it's fair to say that _some day_ we will achieve AGI. It might not be in 10 years, or 1000, but eventually we'll get there.
When we do have AGI, what would that mean for someone like me? Would there be any reason to have programmers (or similar) at all? Would computer science research (or scientific research at all!) exist, or is that something a computer could do?
I think humans might start pursuing more creative/leisure activities, but... I like learning and working!
---
The anxiety of the above has made me feel like it's not worth being ambitious. What's the point of research if some AI might do it better than me in 10-20 years? What's the point of joining a startup or... doing anything? I realize this sounds a bit absurd, but it is something that I've really feared since ChatGPT came about.
You said you enjoy learning and working.
There's a prayer about knowing the difference between what you can do and what you can't, and setting aside the things you can't do to focus on what you can. To me, this seems like the right way to approach this.
And if the concern is "I want to do something meaningful" well... that's a whole other realm of philosophy of meaning. TLDR: I don't think meaningful work is reserved to "influences the thousandth generation after non nonexistence".
Sorry, I put that 10 vs 1000 year generation just to focus the conversation on AGI and not whether or not it's possible in some timeframe.
I personally feel like we could be _very_ close in 10-20 years, which is within my lifetime.
> All humans face the same anxiety of an unknown future. In this way, I don't think AGI is a _special_ concern to worry about any more than when your death date will be.
This is actually very reassuring, and I think a really good way for me to frame it for myself.
And that's the worst case in a way, because a chess computer is a really stupid machine. Getting beaten by minimax is kind of annoying. AGIs on the other hand if they ever exist will be very fascinating entities indeed and I wouldn't mind if they're better at anything than I am. And the same goes for scientific research. I haven't stopped doing math because Terence Tao is better than I'll ever be. How many people can seriously claim to be at the frontiers of science? The value in learning isn't in being the best guy or girl there is.
Also, chess engines did kill chess. Chess used to be a spectacle on the national stage, from 2000 up until Queen's Gambit and the following streamer boom, chess was largely dead.
https://trends.google.com/trends/explore?date=all&q=Chess&hl...
And prior to the explosion in popularity? Barely anyone.
My sense growing up in the 90s was that chess was a niche thing for nerds. I think it seems somewhat more mainstream now than it was then, while remaining fundamentally a niche thing for nerds :) But it seems very clear to me that it never underwent the broad collapse that I remember being broadly predicted after Kasparov vs Deep Blue.
By actually I mean to expend a significant part of the resource (time & money) you own to prepare for it.
AGI sounds like Skynet, the Holodeck, the super information highway and all the buzzwords.
Wake me up in 20yrs, cause I will probably be using c++ and COBOL to sticky tape bank mainframes back together, when the boomers have finally given up on keeping those beasts alive and staving off the thirty seventh crypto revolution.
To put it another way: the enabling tech for iPhone was (mainly) multitouch capacitive displays. I dont think anyone predicted Uber or taxi drivers being at risk because of multitouch when it was unveiled (late 90s?), but here we are.
To paraphrase that old quote:
>First they automated the artists, and I did not speak out because I was not an artist.
>Then they automated the writers, and I did not speak out because I was not a writer.
>Then they automated the office workers, and I did not speak out because I was not an office worker.
>Then they automated me and there was no one left to speak for me.
Almost everything invented in the past 200 years was originally sold to rich people, because they were the only ones who could afford it -- then the price came down and middle-class folks could buy it, then the price came down more and virtually everyone was able to afford it.
Thanks capitalism!
(Unless the AGI will come up capitalism is the best of the possible systems and they will rule them out of the equation)
What's the point of playing chess if Magnus Carlsen will beat me every time we play?
This seems a bit absurd, but it's something I have feared since I learned I am not the best at everything.
Now ask yourself what's the point of playing chess if every chessplayer except you is a copy of Magnus Carlsen.
I think science and math could be the same, which is really exciting! That is, maybe humans would be learning from what the super-human computers are doing, but using that to do different, more human, things.
If the issues as synthetic data conversations like alpha go but for LLMs; setting much longer token contexts (or dramatically cheaper training on custom data sets) And scaling via software and hardware a few more orders of magnitude.
I think then we have some big waves that will wash over any observations we could make at this point in time.
What we have now (so no hypotheticals) coupled with the fact that scaling hasn't yet shown any performance walls makes a pretty good shout that things will probably be different this time.
Humans can't one-shot non trivial planning tasks either. It's the one problem i have with all the papers that try to evaluate planning for LLMs.
Step away from that approach and they're ok.
We are nowhere near generally intelligent software systems.
There could def be bugs I missed tho.
https://chat.openai.com/share/ef77507e-cb75-4112-97f1-a16cfc...
I'll repeat what I said previously in a different way, LLMs are useful but they are nowhere near what is required to achieve generally intelligent software systems. I'm sure they will continue to improve as engineers and companies learn how to utilize them in their workflows but let's temper the hype a little bit because statistical autocompletion is not enough to achieve general intelligence.
Anyways, these kinds of strawmen always baffle me in regards to AI.
Insert random pseudo trivia that most of the general population wouldn't be able to do, see the AI fail at that specific task. "What did I tell you? The AI definitely isn't generally intelligent yet!"
Everybody is out to prove AI isn't intelligent without first defining what intelligence even is. And when other people rightly point out it can do a lot of stuff, they point to some specific task that it gets 90% of the way there but doesn't get perfect and then triumphantly declare AI isn't intelligent. Crazy.
LLMs are cool toys but calling them intelligent is stretching the definition of "intelligent" way too far. It's important to be clear about what the words actually mean because if people start thinking these software systems can be substituted for their own thinking then we end up with all sorts of unnecessary confusion around what they're actually capable of achieving.
And GPT-4 does do this in a couple iterations.
Adding recursion to neural networks has been tried a few times but no one actually knows how to stabilize their dynamics so the industry has settled on feed forward networks with constrained function blocks which have stable dynamics with respect to back propagation of errors.
> What exactly do you want me to state up front?
Exactly what was in your comment that I told you to state up front.
Most people in a discussion about AI and replacing people are working with a definition of general intelligence that includes humans to a large degree.
> This is because LLMs and all neural networks are simply DAGs of function which do not support recursion or backtracking.
Without adding external state I can't solve a sudoku puzzle.
And how many humans can do this? Those poor exhausted goalposts.
It's taken a few short years to go from "keep a coherent story over more than a sentence" to "write a multithreaded sudoku solver in one shot".
Many programmers would fail to do this. I'm not even sure a randomly selected human would understand the question. And I'm wondering if you've setup any loops letting it write tests, search the internet, inspect and debug? If not what you see as an output is essentially it whiteboarding off the top of its "head". Programmers routinely fail to solve simpler things in interviews, and their skills are much narrower than current llms (you can easily argue deeper, but I feel comfortable saying broader).
I did not say anything about threads but I did hint at the fact that coroutines can be used creatively to solve problems which require backtracking and constraint propagation. The fact that this is still controversial means a lot of people are unaware LLMs do not support backtracking, recursion, and constraint propagation. Sudoku is just an obvious example of a problem that is easy to solve if you know about these concepts and next to impossible if you don't. Such problems can also be expressed as integer programs but even fewer people know how to do that so I don't usually bring it up but it would be another good test for any software system that is claimed to be intelligent by the corporate marketing department.
> Such problems can also be expressed as integer programs but even fewer people know how to do that so I don't usually bring it up but it would be another good test for any software system that is claimed to be intelligent by the corporate marketing department.
It's very telling if the level of testing is "can it do something few people in the field can?".
https://gist.github.com/IanCal/9817f8b21b2ea6d77940966ee399d...
>I think then we have some big waves that will wash over any observations we could make at this point in time.
This is spot on. Exactly as how we didn't know what the internet would be, the wiser amongst us realized it was a new and strange era. "An alien lifeform. Unimaginable, both amazing and terrifying." - Bowie. MI will far eclipse the internet.
Imagine yourself in the movie Apocalypto and I'm using a Hollywood film on purpose here - the entire Universe of being as you see it all the sudden explodes its Universe with new Gods/demons different and more powerful _everything_ coming to kill you. And then there's the boring discovery of "we're just one planet in one solar system in one Galaxy...
LLMs gonna be orders of magnitude beyond that (in)comprehension? I mean maybe it will be more powerful and maybe it will come to kill us, but that's not an inconceivable new story line.
Funny that both of us can accuse one another of hubris.
not sure what's not understandable. I think parent is overly hyperbolic. i get that chat GPT is unprecedented, but come on "impossible to conceive of near-term reality" ???
edit: oh! you're the parent. yeah i think you're wildly hyperbolic. There's more drastic historical precedents but of course the future is by definition going to be inevitably more inconceivable over a long enough arc, so I don't think of this as a debate. I just think you're being overly dramatic for effect.
>Imagine yourself in the movie Apocalypto and I'm using a Hollywood film on purpose here - the entire Universe of being as you see it all the sudden explodes its Universe with new Gods/demons different and more powerful _everything_ coming to kill you. And then there's the boring discovery of "we're just one planet in one solar system in one Galaxy...
This is gibberish word salad.
Yes, storage capacity and compute have made major leaps, but it's still only lossily compressing a semantic space. It won't generate things from thin air, and it's certainly not sentient, no matter how far you scale it up.
That said, it may be a useful tool in some cases, and we've also seen its numerous limitations, only some of which may be alleviated in the future.
As always, the problem is not artificial intelligence, but natural stupidity.
This sounds really similar to trickle down economics.
Clothing used to be hugely expensive:
https://www.bookandsword.com/2017/12/09/how-much-did-a-shirt...
> So the shirts of humble servants at Henry VIII’s court cost between 3 and 10 days’ income. That would be similar to someone who earns 10 dollars or Euros an hour spending 240 to 800 dollars or Euros on an item today. (Of course, in the 15th and 16th century, people spent much more of their incomes on food, fuel, and clothing than they do in Europe or European settler societies today, and much less on rent, transportation, and medical care … but it seems that most people could make or obtain one or two new shirts every year or so).
https://www.bookandsword.com/2021/05/08/how-much-did-a-tunic...
> In the Edict, the simplest linen tunic could be sold for up to 500 denarii, whereas a linen weaver was to be paid 20 or 40 denarii per day plus maintenance. Fine linen tunics could be sold for up to 7,000 denarii. Elsewhere in the Edict, workers without maintenance (food and possibly fuel and shelter) earn about twice as much as those with. So a linen weaver would need to work for (500 / 2×40 to 500 / 2×20) 6 to 12 days to earn the price of the simplest linen tunic. That is not so different from the 3 to 10 days’ income for a worker to buy a shirt at the court of Henry VIII of England, considering that the ancients did not have spinning wheels. The linen tunics in 301 CE were probably woven as one rectangular or cross-shaped piece and sewed up the sides and under the arms, whereas the English shirts were cut and sewed from long pieces of cloth, but that is another story.
These days, you can buy a shirt for a much smaller amount of money, so you have more money to spend on other things, and the economy can grow. This isn't a new concept; in fact, it's so old, we tend to forget how things were before it took hold with the birth of industrialization and automation.
It's interesting that in some domains there is not diferencial there is no 10k iphone and you can get a massive TV fairly cheaply. And yes commodity clothing is cheep.
If it was not already obvious we create new forms of value that shift as commodity product become common place or mas market items.
That shift to new forms of value inherently depends on scarcity and that scarcity can only exist as a sum of non-commodity goods, creating new opportunities to fill that pursuit of diferencial value and experience.
It turns out that rich people buy things that don't employ many people, and they buy them from other rich people: high-priced art works, famous jewellery, and other collectibles, existing mansions or apartments at exclusive addresses, and so on. So the prices of those things go up, but no-one else has any more money.
Of course that kind of thinking is also a bit of a fallacy because there is a third way of some people enhancing themselves with AI to level up their own capabilities. These hybrid/enhanced individuals would be able to complement each other and be able to interface with both the real world and the digital world far more efficiently. We'll be competing with our enhanced future selves. Maybe they'll ditch the wetware at some point but the near future is a hybrid of hardware and wetware.
The really fundamental objection to everything I’ve just said is to ask what would happen if we had a system that didn’t have an error rate, didn’t hallucinate, and really could do anything that people can do. If we had that, then you might not have one accountant using Excel to get the output of ten accountants: you might just have the machine. This time, it really would be be different. Where previous waves of automation meant one person could do more, now you don’t need the person.One could argue that AI should only ever be tool to enhance human capabilities, not something to replace humans. Why do we want to do that? For whose benefit?
The article covers this. Is it bad that we don't have teams of people spending their lives pulling barges up rivers or writing out copies of documents by hand? No, it isn't bad.
We fear automation because it may replace our jobs. No worries, we'll make up some new bullshit jobs. Or maybe you can keep your job, and we raise the expectation of your output. It's all "running to stand still", not to mention sustaining yourself becoming ever more complicated and stressful.
I wish we could escape this "job for job's sake" and to always output more in perpetuity. At what point do we have enough stuff and can start utilizing automation to improve people's lives, for example with less work hours, more economic security? This is an area in which zero progress is made in decades, if not negative progress.
This is a S.E. Eisterer (assistant professor at Princeton) talking about the earth shattering innovation that was…. the Frankfurt kitchen. [0]
[0] https://99percentinvisible.org/episode/the-frankfurt-kitchen...
Seems fitting.
I truly believe demand will simply increase to match what can be achieved with AI.
Similarly, I've been using Dall-E to add pictures to the books we read in class.
Compare this to retyping everything at a 4th grade reading level and then painstakingly searching Google for pictures (and usually ending up with mismatched pictures with varying artistic styles or watermarks).
I don't know that we will have true artificial intelligence soon a lá I Robot, but I feel nervous and optimistic about what's to come either way.
By the way, call me crazy but I think the Apple Vision thing is going to be a game changer for business and consumers.
To a large extent, it has demonstrated to me that wading through stack overflow or blog posts or canonical documentation to figure out how to do some technical thing was actually a way worse workflow than I thought it was. But searching "what to do in my town this weekend" or "what are the best books about xyz" remains the best workflow.
The only reason people don't see this is because they weren't around back then. It's not going to be different this time.
There are people who've been around who think it's going to be different this time.
Look it's fine if you don't. Maybe you'll be right and they'll be wrong but this "anyone who disagrees is ignorant" rhetoric is uncalled for. You shouldn't need to result to ad hominems.
I see a major issue for people entering the industry in a decade because ChatGPT/Copilot I can get 10 mediocre answers and verify them in the time it takes a junior eng to respond to PR comments.
I am keeping up with the technology, though, to see how it evolves.
We don't need AGI to eliminate 200+ million jobs. The output of a network driving a car can be literally two signed floating points, left-right/accelerate-brake; in hindsight we will understand there is little intelligence in going from A to B without crashing, a bee does it, in 3D even. To eliminate 1+ billion jobs you take the driving network and give it hands: flipping burgers, feeding a CNC machine, handling packages, all these jobs will be gone forever, as they should.
The only question is what will politicians do when their solutions so far have been to raise the retirement age.
Boston Dynamics' Spot costs $75K today and Atlas is around $150K, once the price gets closer to $10K there is very little reason to ever employ persons as assembly line workers, construction workers, warehouse workers, machine operators, truck drivers, janitors and cleaners, agricultural workers, security guards, food service workers, garbage collectors, and so on.
And even if 3 billion people could be forced to do spreadsheets to "earn" a living, "by the sweat of [their] brow", that would be an even sadder world than our currently sad world.
My argument is that we need a new metaphysics for what work and life means, one where the right of a person to have food and shelter is not tied to the economic value they produce. But I have no hope for this world: the trillionaires of tomorrow will share even less than the billionaires of today.
I'm seeing people migrate perfectly fine ASP.NET or JSP applications to Salesforce just to get away from their internal IT teams, which can stretch out the paperwork to 2-3 years just to spin up a VM for a legitimate purpose.
Some sales asks were legit and we got 'em done. Some were legit but low ROI and never done/priority even as sales had majority say in those priorities. Some were "please take this customer-hostile, flaky, human-explicitly-in-the-loop CRM workflow that has the thinnest veneer of plausibility above pure gaming of KPIs and insert it into a core product flow that's working well".
I think low code tools will continue to proliferate and I'm not against it. I do think they will be an interesting test of how much and in what contexts consumers are willing to accept half baked solutions. The capital environment this past decade has meant corporate leadership has been able to touch hot stoves without getting burned. Often they've been rewarded for it.
Yes. Just like search. But that didn't mean that search was replaced by a bunch of radio buttons and click boxes. On the contrary, we learned how to use freeform search. And we will do the same for LLMs.
What I'm seeing is that the systems are most powerful when driven through and conversational interface. They are strong translation systems. They can ask questions and incorporate the answers into their outputs. The way to use them is only clear to a few people who are deep in the experimentation process now, but it will become common knowledge and their outputs will improve to the point that what we are doing with them now will seem primitive.
Just to illustrate by way of one example I've had bouncing around in my head: imagine a zotero-like interface for accumulating tagged information, where it's both built from pulling out information from conversations and can be explored conversationally. I want to click a button that says "show me all the information I've accumulated about LLMs (or whatever)", which includes both links and conversations I've had about it, and I also want to be able to say that same thing to the conversational interface.
In truth, I think we already have examples of this paradigm, in the various "copilot" interfaces, where you can do both things.
We do not want to read tweets or content created by AI's or machines, we want to connect with humans who can inform or teach us better. Maybe this would mean the liberation of humans from physical fields to digital fields where we keep plowing.
We were meant to connect, discover, roam, invent and innovate and instead ended up somehow sitting in front of a monitor and calling it work!
However replacing someone or an entire role category is a difficult transition. Humans don't change that fast, including the part of replacing other humans.
It's more likely that certain jobs will be in lesser demand when the businesses in question can get by using LLM, for filling a new position, rather than firing someone and replacing them with LLMs.
Here is an article on how recent historians believe that English male height during the Industrial Revolution went down due to malnutrition:
https://www.economist.com/free-exchange/2013/09/13/did-livin...
Historians now think the Industrial Revolution was a giant economic boom where the majority of individuals actually became poorer.
Here is an Indiana University paper, consistent with the findings of others, that automation did reduce manufacturing jobs
> The basic takeaway from the Ball State work is that both forces—offshoring and automation—are in play in the massive occupational realignment due to hit the U.S. economy, but automation is dominant (Wells, 2017). This assessment, that automation is the greater threat, is consistent with their earlier work placing the loss of manufacturing employment on increases in productivity
https://www.ibrc.indiana.edu/ibr/2018/fall/article2.html
This was all around the Trump years when the media and academia were happy to say “China didn’t do it”. Even the NYtimes said it at the time.
https://www.nytimes.com/2016/12/21/upshot/the-long-term-jobs...
Yes we can find jobs for people if we keep trying. But the jobs that we find in a post-scarcity services society are often precarious and meaningless - and people know it. Agriculture and manufacturing, which are the two job categories that really matter, do lose jobs to automation because we are squeezing them for profits all the time through all available methods.
To say that everything will be hunky dory with AI is just the same as saying the steam engine won’t take your job. That was BS in the 1800s and it’s BS now. And yes people did lose their way of life and they did become poorer, as demonstrated by the current studies on the matter.
I think the assumption underlying this is that somehow stable on the other end, but I don't necessarily see post-industrial service oriented societies as stable economic realities. Having lived in one(UK), I found it as close to a capitalist dystopia as one can get.
Children are ranked by intelligence at a very young age, making it so underprivileged kids are told “you’re not going to make it” at a young age and relegated to a “lower” class. London street drug sales are thus dominated by disillusioned teenagers. Entire cities lost their way of life when Thatcher ended their industries and became dead ends where heavy drug addiction is quite rampant. The population is parted into two (as is common in “post-industrial” societies) the folks who take the now exorbitant loans for education and “make it”, and those that don’t and get relegated to receiving a living wage for the rest of their lives. There is an almost palpable feeling of haves and have-nots, around housing and increasingly everything else.
Yes there is some social contract in place but the post-industrial society is not a gentle one that I’ve witnessed. I’ve seen 2/3 (south of Brazil, north of Portugal and industrial England/Scotland). They don’t really compare well to successful industrial countries like Germany.
Vaclav Smil said “with no manufacturing there is no middle class”.