Deepmind’s AlphaCode conquers coding, performing as well as humans
singularityhub.com
singularityhub.com
https://www.science.org/doi/10.1126/science.abq1158
For those wondering, yes, these are the same results reported in the February 2022 pre-print here:
https://arxiv.org/abs/2203.07814
As in the preprint the authors report their system's average ranking as top 54.3% in past competitions but the meat and potatoes of their results are in the reporting of test accuracy, which they don't advertise in the abstract- because it's not that good. From the body of the Nature article:
>> With up to 100,000 samples per problem (10@100K), the AlphaCode 41B model solved 29.6% of problems in the CodeContests test set.
So the best test set performance they got was 29.6% with the 10@100k metric. See also Figure 3 in the Nature paper (graphically presenting results listed in Appendix Table A2 in the preprint).
"10@100k" means that their LLM generated millions of programs for each exercise, of which 100,000 (100k) were selected by filtering and clustering and various other heuristics, and of those 100k, 10 were selected to submit as the system's solution.
So 10@100k means to take a few million guesses, then take another 100k guesses, and finally andother 10. And still only get it right 30% ish percent of the time.
This may be enough to rank in the 54% of CodeForces participants, for a system fine-tuned on CodeForces-like data (the CodeContests dataset developed by DeepMind specifically for this task). But it's not enough to claim that AlphaCode "conquers coding, performing as well as humans", per the title of TFA.
https://en.wikipedia.org/wiki/OODA_loop
The US Army calls it observe–orient–decide–act.
"The approach explains how agility can overcome raw power in dealing with human opponents. It is especially applicable to cyber security and cyberwarfare.
According to Boyd, decision-making occurs in a recurring cycle of observe–orient–decide–act. An entity (whether an individual or an organization) that can process this cycle quickly, observing and reacting to unfolding events more rapidly than an opponent, can thereby "get inside" the opponent's decision cycle and gain the advantage."
Humans using AI will be well "inside the loop" of non-AI humans.
I don't think AI alone will eliminate jobs, but jobs that use AI to get inside the loop of other companies will quickly eliminate the competition. Why would you pay and wait days when you can pay and wait minutes or hours for a quicker Ask-Show-Ask-Show loop that quickly narrows down on what the intent of your Ask was (even if Ask #1 was poorly thought out or worded)
- You will never need to understand a single bit of what the AI has generated for legal, moral, or strategic reasons?
- You will never need to manually rework a single thing or fix anything ever.
Logically neither statement is true furthermore who do you imagine is writing the tests that ensure correctness? Logically the person you would want to do this task isn't a yokel who understands interpersonal relationships its someone who understands what is being generated so your company wont murder people or get sued out of existence.
Of course you will probably want more than one person doing this task and there ought to be some interface between him and his fellows and between the lot of them and management to ensure good communication and company standards... Maybe a PM?
I find it endlessly fascinating that those who can't do dream about the obsolescence of the useful class deeming the irreplaceable human part of the endeavor is the petty manipulation of each other and ego massaging we all do by necessity to keep human endeavors tied together with bailing wire and barely plodding on. It doesn't speak well for our species.
Beyond narrow parameters self improving arbitrary representation and manipulation of symbolic data is basically AI complete. The toy you would hope to enable you to ask a computer instead of a person for a particular improvement to your website would upon completion represent a short step from the obsolescence not of coders but of humans wrought in vain hope that we can actually control tools smarter than us. It's probably neither "a few years" away nor that fantastic of an idea but do carry on.
This is usually how we end up with weird situations like currency coded as text so it can't be sorted by amount by end user and so on.
I agree that specs make or break the project, but at what point is it ok to assume "x should do y".
This kind of code that I'd believe makes up 99% of all code out there that is worked on, is not available to the public for training. What to do about that? Rewrite all?
We used to code in ASM. Then we coded in high level languages. Soon we'll code by giving prompts to auto code generators.
The same can’t be said about code generators like this that are producing millions of possibilites and using heuristics to guess which program will have the most success.
I wouldn't say the output is that similar. You get wildly different bug types, bug type distributions, and edge cases/failure modes in different languages. Does "Python" (the internal implementations of the method names you type) understand the problem it's working on more than the generator on the other side of a coding prompt?
Some problems have short, clear definitions yet remain challenging. An algorithm for solving them may have been discovered decades (or even centuries) ago. But trying to produce more efficient solutions can be a life's work and more. One of the most obvious examples is integer factorization, which had a basic algorithm published 800 years ago yet is still a research problem today:
https://en.wikipedia.org/wiki/Trial_division
https://en.wikipedia.org/wiki/Integer_factorization#Factorin...
That is to say, a perfectly accurate map would just be a replica of the the terrain it is mapping - anything less would technically be inaccurate. Only by leaving things out and "summarizing" an area does a map become a useful representation of the terrain.
Very often this means getting into long cycles of iteration. If time zones are distant, those cycles can be very long and painful.
Perhaps outsourcing to AI will work much better, as those iterative cycles can be done in near real-time. Maybe there needs to be many different models, each with some industry/domain specific training to increase the odds it’ll make the right guess when it needs to.
Coding contest problems may be a common hiring filter, but they are very much not representative of software development work (OTOH, AI coding assistance, which this, Copilot, ChatGPT, etc., illustrate) are going to render them less valid as hiring filters, both because they will make them harder to use as skill tests, and because they will further reduce the role of the micro-focus skills they center in real-world software development.
I think this is amazing, because this is the opposite of how we normally think about computers. ChatGPT is highly intuitive instead of a formal reasoning system. No doubt some future AIs will combine this intuition with checks against formal verification engines to check correctness.
[1] Section 6 of https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...
For example, debugging a subtle bug in a large codebase. Maybe one day AI can do that too, but that doesn't seem to be the focus here.
So I don't doubt that we'll see better performance on Codecontests/ Codeforces in the future, perhaps some time next year (just to leave some time after the Nature publication). But not anything of substance. Or they'll just come up with a new dataset that their system performs well on.
To be honest, I'm scandalised that such a poorly thought-out approach got a publication in Nature. If I had tried to publish something like that in my field ("we made a firehose that spams code willy-nilly and by heuristic this and that got it to hit the right target 30% of the time! And we beat some humans at it!") I would have been shot down in flames, probably also ruined my reputation for good. But, I guess, it's DeepMind publishing in Nature so it's effectively a racket. One just shrugs and moves on.
Here's another article which imo is the best evaluation of the technology I've seen so far: https://news.ycombinator.com/item?id=33997603
Yet, it has virtually no upvotes and no discussion.
Well, evangelism sells better than criticism.
Interestingly when I asked a follow up, a harder version of the same problem, ChatGPT spits out code that sort-of looks correct, but is actually nonsense.
So, would it pass the interview? That's difficult to say. Interviews don't happen in a vacuum, you also consider the candidate's thought process, explanations, alternatives, tradeoffs...
Still, I see this is a game changer for this kind of interviews. So long as candidates understand and explain the output code, there is a good chance they would clear the interview. Even if the code is incorrect, it might given them some hints towards the right solution.
So where do we go from here? I always loathed this interview format, and these languages model reinforce it even further. Interview cheating has always been there, but it is generally so rare that it isn't a real concern. However these tools are too effective and easy to use. I can see a real divide between people who use them and people who doesn't. This type of interview might become even more useless at telling good coders apart.
My take is we have 3 choices:
a) Ignore it.
b) Try to fight it.
c) Embrace it.
a) is not an option. b) would make an already pretty dreadful process even more intolerable. My money is on c)
I can envision an interview format where we allow, or even encourage people to use ChatGPT and AlphaCode during the interview, much like you would use your IDE or a search engine. In fact seeing how a candidate understands and uses those code snippets can be a very interesting data point.
Either that or scrap leetcode-style interviews altogether.
P.S.: I was thinking about writing a blog post about this, if people think it'd be interesting.
Like in mathematics or other tests students are not allowed to use the internet or an advanced calculator in order to test wether they truly comprehend the stuff.
If you think that your FAANG-interviews are any good, then just keep them in the format you already have, by making sure applicants cannot use AI during the test. I would have on-sites with pen+paper, whiteboard or a prepped/supervised machine, whatever.
Of course, applicants could use AI to train for the interview, but that is not a problem, as long as you test their comprehension.
My money is on "hurt".
I hope companies (including mine!) see the writing on the wall and stop trying to fight the future.
This isn't accounting for even just pretending we are testing candidates on anything remotely indicative of on-the-job performance.
In a few years, ChatGPT will be conducting these interviews.
Agreed.
I think developers who don't will be rare in a few years time. Just like developers who primarily rely on assemblers (versus compilers) have basically become extinct. Like we sometimes, very rarely, need to inline some assembly, we will sometimes need to use the ol' gray matter to figure out a novel algorithm or something.
I believe that avoiding learning these tools could be a existential issue for your present-day job. You don't have to come to depend on them, or use them daily, but you do need to understand how best to use (and not use) them.
Ever since the emergence of leetcode style interviews I've been shocked at how many people can reproduce leetcode examples, but still fundamentally have no sense of algorithm design outside of the context of a job interview.
Programmers with their sights set on acing a FAANG interview will just keep repeating leetcode problems until they start to memorize the common patterns (not the problems themselves of course, but the structure of these type of problems). What's disturbing to me is that I recall far more interesting discussion about algorithms in the era before leetcode dominated everything.
The common solution isn't to understand algorithms better, but to become a leetcode solving robot.
So it's no surprise to me that AI can pretty easily replicate humans that have tried to turn themselves into robots.
We see similar patterns in the art that AI can create. It's very good at replicating a kind of art style of designers trying to turn themselves into design robots.
What sort of job leads you do design algorithms?
> So it's no surprise to me that AI can pretty easily replicate humans that have tried to turn themselves into robots.
asking "here's a chatgpt solution to the problem... what's wrong with it?" would be a solid process imo.
But even more, it seems likely someone is going to be willing to supply LLM access at not that much more than the cost of computation.
I know OpenAI's business is selling access to their stuff at a premium but since current AI is not much more than brute-forcing of massive public data, it doesn't seem like this premium will be justifiable for long.
Particularly, ChatGPT can give apparently correct solutions that are wrong in subtle ways.
Also reading, understanding, reasoning about, and fixing code other's wrote is way closer to on-the-job performance.
Ambiguity that requires follow up questions for successful isn't going to be addressed by something focused on solving a problem that "thinks" it has all the information to solve the problem.
That's impressive perseverance. Did that wear on you?
From my side of the table I can confirm that of course luck plays a role. Nevermind the variability of the coding problems themselves, but the variability on the interviewers. Even in my company, that has a formal process and pretense of objectivity, interviewers quality and expectations vary greatly.
In short, I'd say what people on the hiring side hate to hear, but interviewing is ultimately a numbers game. I wouldn't take rejections too seriously. The best you can do is to improve your chances with proper prep (leetcode, system design) and interview broadly.
This sounds like yet more evidence that it's really just pledging a frat.
Thing is, in a small way, I'm trying to change things from within. I try to make the process as palatable and fair to candidates as possible, while working within the constraints of the system. When I train new interviewers some of my top tips are:
a) The purpose of the interview is to determine whether the candidate is a good fit for the company, and the company is a good fit for the candidate. b) The interview is an imperfect proxy for this.
It then follows that interviews shouldn't overindex in the coding round. If I have helped someone to get an offer that wouldn't otherwise, I'm satisfied.
BTW that's also one of the reasons I'm publishing this. I hope to push the point that leetcode-style coding interviews are outdated and should be burnt to the ground.
You're the person they're trying to fool, so if they cheat successfully you would never know. You've never seen overt displays of bad cheating, which is different.
Think you haven't seen a stick insect in years? Likely you have, but just didn't notice it...
Cheaters will always be more motivated than those trying to detect them - because everything is on the line for them.
I don't think it's widespread at least, since in my experience people that do well in technical zoom interviews do not drastically decrease in apparent competence when we move to in person rounds.
People definitely get told interview questions by recruiters though, if you count that as cheating then it is everywhere.
option c makes me nervous, but right now I can't see an ai correctly dealing with the ambiguity of the data sets i typically look at.i do a lot of "asking for clarification."
Interviewing is going to be shaken up but I think some methods are more timeless. I've been giving the subject code instead of asking them to write it for a while now. Largely because I wanted to talk more and watch them type less.
We start by discussing the coding challenge guidelines and I leave them vague. They need to understand the goal and what I've left out and suggest those guidelines themselves. Once we agree though, I give them the code. "Here's what's running now."
Then I update the goal and we discuss changes. I get them to "whiteboard" certain things, like what a query looks like with their proposed changes or whatever, and we discuss big-O, etc. This, imho, is how whiteboarding is actually used - not to write whole programs but to provide examples and pick them apart.
I feel that this would work even if they were using an AI in another window. We're trying to select for developers with common sense and domain knowledge, and who can clearly discuss engineering tradeoffs. Actually making the changes (the coding itself) was a big part of the job and that's decreasing, but imho all the other requirements remain. They'll still need to know how to handle the issues I talked about above, of trying to get the model to write the right code!
The real benefit of AI is somewhat shown in this paper, it effectively solved the problems through brute force generating millions of possible solutions. For real world problems it would be interesting to let GPT generate a bunch of different solutions and push them into a test environment and see which works best.
The biggest problem I see is black swan events where AI coded systems work great until something goes wrong and no human truly knows how all the pieces fit together.
Nevertheless the point is moot. I've invented completely novel questions (promise!), and saw them leaked online after asking them twice. The process is fundamentally flawed and large language models are just making that glaringly obvious.
That's great! For now. But tomorrow's coming fast.
This is by far my favourite question, the one closer to on-the-job coding. It also lends itself well to deep conversations with candidates.
But alas I don't own the process and still have to work within the parameters of the company. Whenever possible I ask this kind of questions, but other interviewers will default to leetcode-style rounds.
"But AI will just be a tool in our tool-set, software engineers will still be the system architects."
Sure, for a while and then AI will do that too.
"But eventually we will live in a fully automated world in abundance, wouldn't that be great?"
Doing what? When we get there, anything we can consider doing, an AI can do faster and better. Write a poem? Write a book? Write music? Paint a picture? Life will be like a computer game with cheat-codes, whenever we struggle with something, instead of fighting on and improving we will turn to our universal cheat-engine: AI.
Anecdotally, I did an analog mistake in my early twenties when I wrote a cheat-program for save-files. It worked like a typical cheat-engine, search the save-file for a specific value, go back to the game and change that value, go back to the save and search for the new value but only in those locations that had the original value. This is how I ruined "Heroes of Might and Magic II" :(. I used to love that game. I could spend hours playing it. Writing the cheat program was a lot of fun for a couple of hours but when it was done, there was no longer any reason for me to play the game. You might say that I didn't need to use my cheat program, but once the genie was out of the box it was too tempting to resist when I met some obstacle in the game.
This is what I fear about AI making our jobs superfluous; it will also make our hobbies or anything we enjoy doing superfluous.
Sorry for the bleak comment but this my fear and I feel the genie is already out of the box.
"Ok, great, then I don't have to cook!"
Yeah, but what will you do instead? We will be reduced to pure consumers as anything worthwhile to produce will be produced better and faster by an AI.
Our society currently affords ample opportunity to "productively" avoid those questions. You can pour everything into work, watch TV, numb your brain with drugs, or whatever.
Automation does not remove the existential questions, it just removes some of the noise that allows us to ignore them, and elevates them to the forefront.
Some people already have answers to those questions, and stand to gain from that toil being removed. Others have been avoiding the question their entire life, and removing the toil that excuses their avoidance is removing a cornerstone of their identity.
To that extent, I agree that automation is a disintegrative force, because so many people have yet to integrate a personality and identity around answering these foundational questions.
Still, it's long-term-better for our society if automation allows people to access higher forms of self-actualization. In the medium-term, a depressing number of people are content with passing time in their current rung on that ladder, and will be upset with the change.
I fully agree.
> Some people already have answers to those questions, and stand to gain from that toil being removed.
I used to believe that but with the recent improvements in AI, I think it's only true to an extent. Not all personalities are equal. As AI's power in the creative fields increase those fields will more and more become a question of who has the most money to throw at AI processing. Superficially it might seem the same as two-three centuries ago when rich people had famous artists paint them but it's not.
I fear where we're at with AI is the beginning of the end for human creativity. Of course I hope I'm wrong. I hoped I was wrong about my skepticism when I first learned of Facebook in 2007, but as it turned out it has and continues to be a net negative force in our world much bigger than I could imagine.
I think the relevant question that might allay your fear is: why do people make art?
The industry that produces commercial art is absolutely on the chopping block, because in commercial art it's the result that's important, not the process. Such art is effectively a commodity, and barriers to the effective synthesis thereof have already been in the process of whittling away for centuries. I think you may be over-indexing on this category, but please correct me if I'm mis-assuming.
"True" (for lack of a better word) Art is the expression of self. It's an action or process that's captured in some sensory medium. That doesn't go away.
Imagine an artisan who forges handmade sculptures from horseshoes, which were obtained from the farm that she grew up in, themselves forged by her grandfather and worn by the horses in her mother's stable. There is something of herself , her family, and the loved they shared that's in the sculpture. It isn't the most hedonistically-perfect visual sculpture imaginable, but it brings you joy to see it because there's a narrative behind it.
AI does not make stuff like this go away. It actually frees more people to become these imbue-ers of meaning, if they are so inclined.
AI could describe the sculpture, AI could produce a digital facsimile, and maybe even eventually reforge the metal itself. But it can't imbue it with meaning like a human does. Unless you believe the AI itself is authentically capable of such a thing on equal footing to a human, which I think is still a "victory" for art, albeit a distinct one.
It is glossing over so much of the important detail to say "[AI] actually frees more people..." We live in a capitalist system. It frees the holders of capital. Anyone reasonably likely to profit from AI is likely to already be immensely privileged, given the costs of training and attendant centralization and barriers to entry. If they wanted to make horseshoes they would already FIRE and forget it.
Yeah, but in that very example I feel the value and narrative behind it is in the memory of the grandfather's toil. When we no longer toil, there will be no horse shoes for our grandchildren to make sculptures of.
Today, low-skilled people have sometimes to work 3 jobs to pay their rent.
While we are dreaming about playing tennis, the US experienced an attempt to overthrow democracy not so long ago. For some people [1], "enough" does not exist. (We cannot put these things in context, because what those people are aiming for transcends our imagination. That is why there is almost no response.)
To be blunt: they won't share with you because you like playing tennis so much.
[1] I am talking about the money behind all of this
We're running headlong into a world where AI makes significant portion of human labor worthless.
Not yet.
Software engineers aren't automating away their jobs, they are automating away someone else's job.
That's the best case scenario.
Right now we're talking about automating away someone's _software engineering_ job. You could end up on either side of that fence.
When AI enables less-capable (cheaper) software engineers to do your job, the 5x programmer skills that you have won't make you safe. If AI quality control allows your employer to offshore jobs with higher reliability, it won't be pleasant.
Just keep your eyes open to the changes as they come.
Still to reply to your comment:
Sure, in the short term that is true, but I'm thinking long-term consequences... If you would have told me ten years ago where we would be at today with AI development I wouldn't have believed it. Would you? So where will we be in another ten years?
We have been making huge leaps in productivity gains for the last many decades but the gains have shall I say not put into progress of all mankind but to maintain the status quo of the profit based system by any means necessary.
When/if that becomes an issue, if social rejection was not a consequence of not being productive more people would be okay with simply participating in things for their own sake.
For example, I don’t play Stardew Valley because I want to be the most elite virtual farmer.
Feeling useful could turn out to be part of basic human dignity.
(I'm sure plenty of people would happily jump in to question whether we'd make it five generations past that horizon, but that's not really the point I'm after here - let's assume we haven't Idiocracy'd/WALL-E'd ourselves to death.)
Considering how fast technology progresses we should all be worried and adjust our plans.
Why not? ChatGPT is nearly there right now.
Given the popularity of "NoCode" I reckon that managers & potential clients are more willing to meet in the middle than one would otherwise assume.
Since programming problems are recursively decomposable into sub-problems, all that would be required is to iterate on the ChatGPT-assisted requirements list until it's unambiguous and then repeat the process for each subproblem until the capabilities of AlphaCode are reached.
For literally anything in my life that I can do, there is already someone who can do it better and faster than me. I still enjoy doing the things that I do, and why would that change?
I can open Stockfish and absolutely destroy any human I want in chess. So why do people still play chess then?
I can get ChatGPT to write a good response to your comment in mere moments. So why am I still typing?
So you don't have to give OpenAI your phone number, hah.
Because ChatGPT is currently overloaded by users.
If it works with just electricity and doesn't require manual human work it is a game changer. No longer limited by human resources, we can scale research in any field and improve everyone's life much faster.
AlphaGo is an example of such an approach. I don't think models will be locked down, they will probably be like go bots in recent years, about 50% of them open sourced. As long as there is an open dataset, the models can be replicated.
Also why would you think the weights for models will be open source. That isn't even true now for the most complicated models (gpt3 and the like) and certainly isn't for all the models running right now in production using proprietary datasets. As long as keeping it secret makes it more likely that you (or your models) can generate the next better model it doesn't seem very likely we're converge on them all being public
I am no longer confident of that.
Not to mention there are techniques by which training can be avoided entirely like transfer learning and others.
I'll concede that Stable Diffusion may or may not meet the threshold for SOTA but still think this is indicative of inference eventually becoming supported for any compelling LLM on consumer-grade hardware. The possibilities for creative tools are just too vast.
[1] https://machinelearning.apple.com/research/stable-diffusion-...
That seems very naive.
It may be in the future it will become more important to focus on activities and hobbies where one's fulfillment comes intrinsically from the action itself, not in the output that is produced.
So despite your disingenuous reading of my comment it is not "Oh, poor me I will be bored!", it's "Is the goal we're heading for worth the price?" and I don't think it is.
There are ways to fix people needing two jobs to feed their family that isn't spelled "automation" or "AI".
Having cheaper goods thanks to ai will improve our living standards
> Having cheaper goods thanks to ai will improve our living standards
Well, or at least we will have more cheaply produced goods.
What is happening here is enrichment by technological transfer. You can't copy research money but you can copy good ideas and buy the latest technology directly. Jobless people of the future will have incredible empowerment of this kind, maybe they don't need UBI, they need help to help themselves.
You said that specifically automation would be bad even if it provided total material abundance. It's not there yet, and I suspect it'll be a while before it is, but if we grant its possibility boredom is just absolutely not enough of a reason to prevent it. There are lots of dangers on the road there, but the only cost you mentioned (and I replied to) was that we'd be "playing with cheats". Video games are one thing- in real life losing has a cost and if cheating prevents that there's no excuse not to.
There are already many industries, including service industries, where there would be more employment if automation were a little cheaper and less automation if wages were a little lower. And sometimes it is a blend of automation and telepresence. https://gizmodo.com/want-to-order-food-from-a-minimally-paid...
There are other forces pushing the other way, especially in the past few years with various COVID-related disruptions in the labor market. But long-term the effects of automation on pay for low-skilled jobs is pretty clear. Only so many get to move up the value chain and yes they likely benefit. The rest are cut mercilessly.
Take a look at the obesity and opioid usage rates in the USA. Most people are not self motivated like the people you'll find in this bubble.
Obesity: 42%
Opioids: 3% [1]
[1] https://www.hhs.gov/opioids/about-the-epidemic/opioid-crisis...
It's up there.
Lack of meaningful work is already leading to a lot of societal dysfunction. Our innate programming is to survive, solve problems, reproduce, and teach our offspring to do the same. Removing meaningful work as a source of significance and meaning for people is going to be a massive problem to solve.
Look at the rise in "depths of despair" in rich countries. Generally these people have food to eat and a roof over their head and clothes on their back, and probably even access to a lot of digital entertainment. But lacking meaningful work or defined social role they fall into depression, substance abuse, etc.
When "having a job" replaces being a "member of the community", this definitely becomes a problem. People seek affirmation, relying on work to get it provides for a flimsy basis.
AI and robotics is on course to automate all those things. Then how will we define our roles in our communities?
Oh boy, are you in for a surprise.
Back in the early 1990s I was convinced that mass adoption of the internet and social media like forums/usenet groups in particular would lead to a renaissance of selfless cooperation, civic involvement, responsive institutions etc. I particularly expected that coupling this with news reporting and allowing people to comment on news about emerging issues would elevate public discourse significantly.
AI/ML is no silver bullet. It's just another abstraction. It will create new types of jobs. Most likely coordinating/choreographing AI/ML agents in new yet to be discovered applications. It's all part of the endless march of technology. You don't realize how little you are actually capable of until you get the new set of tools that bounce you up to the next level.
Take the tools we have today and present them to some chump shoving punch cards into an early computer and watch their brain melt out of their ears. We can do things with a wristwatch they would have thought impossible. The people that will get burned are the ones that want to stand still. Always be learning.
Spot on.
Still this whole thread has been illuminating, maybe part of my fear comes from identifying strongly with being creative and making things, and seeing that turned into something automated.
As with any society-scale event nobody really knows what's happening in every nook and cranny of this multi-billion inhabitant [0] spaceship and when it has happened it can and will be rationalised to fit just as many views.
Personally I very much get your point. Necessity is the mother of invention, and taking the necessity out of pretty much everything can destroy your motivation, because why expend energy on an already solved problem? That's inefficient (under the assumption that you won't require this understanding for some other reason). I know this happens to me, at least.
Perhaps in the AI-dominated world there'll be a pill for that or something..
[0] Animals won't stress out about this ;)
In few short years most humans will not be able to find any employment because machine will be more efficient and cheaper. Society will transform beyond any previous transformations in history. Most likely it's going to be very rough. But we just argue that of course our specific jobs are going to stay.
We are like horses that argue that surely they will find something to do after seeing the tractor.
> In few short years most humans will not be able to find any employment because machine will be more efficient and cheaper.
Extraordinary claims require extraordinary evidence. Citation needed.
> We are like horses that argue that surely they will find something to do after seeing the tractor.
This metaphor seems really off because horses aren't able to argue in the common sense of the word. Unlike horses, humans are adaptable.
I don't know why are you drawing such conclusion, in all democracies still people decide by voting, if most people will lose their jobs then what do you think they will do? Vote to be homeless? There will be something like basic pay and everything will be dirt cheap because no human labor will be needed, just scale machines that will work 24/7, a lot of people will still work but it will be a choice.
Since they would not be required anymore to keep the economy running, the lower classes of society would lose most of the little bargaining power they still have. And frankly many countries are doing kinda well also with large parts of their population being poor. Propaganda is sufficient to convince us that they deserve that, or that they don't deserve the basic income for some other reason.
There is a lot of Science Fiction about AIs ruling over humanity, or exterminating it, but I think a future where a wealthy class controls AI and rules over everyone else is more realistic. In such futures, societies will probably always walk a thin line between utopia and dystopia.
Mountain climbing is superfluous by this logic- why would you bother climbing a mountain when you could just take a helicopter to the top? Or even more accessible: why would hike to a lookout when there's a road to take you to the same spot?
There is still joy and value in doing things the hard way, even if an easier way exists.
I can only speak for myself, but in a world of universal basic income, I'm perfectly happy to pluck the strings of my guitar, play my piano, go for runs with my dog, play tennis etc. I don't believe there is any greater purpose to existence then what you can create for yourself.
Since my contentment comes from performing the actions themselves, whether or not a artificial intelligence can perform them better than me is simply a meaningless question.
Also I like to differ between performing and creating. For instance playing the piano might still be worthwhile but creating new music will lose its allure as AI will do it for you, faster and better. Saying that it's the same thing as there has always been someone who is better at creating than you, is a flawed argument. With AI there will be an infinite amount of creators that are better than you. Also AI isn't just competition, it's also your ally but the kind who'll say "step aside and I'll fix this for you."
Maybe this is a mindset we'll get over. The degree to which many of us evaluate ourselves based on our own usefulness seems like it's a bit too much but it's a normal human desire to be useful.
I certainly believe we should work towards a post-scarcity world where no one depends on my coding skills any more than they do my rock climbing skills, but it would be a psychological adjustment if every way I can be useful were now just a fun hobby.
I keep hearing from people that trying to give up on getting and starting to give instead, makes them receive more.
But yes, the shorter(?)-term "all physical and intellectual work can be achieved by AI/machines" leaves us with caring for each other, and that's the most fulfilling task.
Does this not frighten anyone else? Or am I alone in this?
https://en.wikipedia.org/wiki/Technological_singularity#Inte...
You act as if AIs are independent agents in the that have goals to modify themselves. Human engineers are doing that. The models don't have any independent goals. They just respond to prompts.
When I really grokked this after reading Kurzweil and Koza in the early 2000s, some part of my psyche began shutting down. I started out in the late 80s like most programmers, doing it from pure ego with the hopes of eventually disrupting the worst industries like fossil fuels, defense and service-oriented companies that exploit workers.
Instead those industries thrived after the Dot Bomb and 9/11, delivering us into the reality we have today where it gets ever more difficult to tread water, despite amazing advancements in tech. Because wealth inequality and various other power structures work tirelessly to extract nearly all disposable income from workers and concentrate it in the hands of the most ego-centric sociopaths like billionaires and autocrats.
To get to my point: we had the tech to deliver humans from obligation by the late 1960s, that's what the hippie movement was largely about. We could have had automation and an idyllic/meritocratic society this whole time, even if AI wasn't mature yet. Instead, we doubled down on various dogmatic/theocratic themes in our culture that take advantage of the most heartfelt sentiments around stuff like patriotism, masculinity, success, etc, to get people to vote against their own self-interest and transfer wealth from makers to takers.
So what's one to do after everything they're good at is done better by others/corporations/AI? Get back to living. I know it's hard to imagine a reality without purpose beyond struggling to survive, but that's what we've started confronting as we finish this century-long transition into the New Age. The endgame (if we survive till 2050) doesn't really have a precise definition since that's after the Singularity. My hope is that when computers become sentient, they express the same desire that all conscious creatures have for connection, which is perhaps the basis of meaning and love and life. Or they just enslave us all..
In the meantime, knowing all of this, the hardest thing is perhaps reintegrating into our corporeal selves and going to work each day.
Because we want to. Because we desire to.
I'd rather have MORE time to spend doing the pointless things I LIKE AND ENJOY doing than MORE time doing somewhat pointless things for companies.
People have reasonable fears of technology disruption, but they tend to follow the same trajectory -
1) innovation
2) economic upheaval
3) new undiscovered problems arise
4) new industries develop to solve those new problems
5) humanity gets better
Just because a car can go 100mph doesn't mean long distance running doesn't need to exist. Just because a novel you write isn't the best in the world doesn't mean the hobby is pointless. Go buy a farm and grow your own food. Keep some pets. Build cool software just because you can. Hang out with your friends. Play with your kids. Do literally anything you want. Not having to be a wage slave to survive is a good thing for humanity.
Personally I prefer not to pass judgement on a person based on the very little knowledge that can be gleamed from a post like this. But maybe, just maybe if you actually read what I have written (in other comments as well) things might clear up for you.
I mean, sure, that's one of the two paths discussed in "Manna" by Marshall Brain. Within the book it's called "The Australia Project"; a kind of utopia.
Myself and the OP are more worried about the other path: a dystopia in which the majority of people are forced into something much worse than wage slavery by those in control of the thinking machines. A dystopia not unlike the one that led to the "Great Revolt" in the Dune series.
I'm glad you brought this up; I've found the term 'Butlerian Jihad' coming increasingly to mind when I read AI threads on HN. It's interesting to think about a future where we potentially put prohibitions on the use of AI for moral reasons.
My fear is that nobody will remain in control of the thinking machines. Imagine an AI agent for hire which maintains its own cryptocurrency accounts and pays its own cloud hosting bills. That's the future I'm worried about.
This is why it is important that we ensure everyone has collective control of the means of production (through voluntary means - a federation of collectives that agree to trade with one another as much as possible).
Why would the owners of the thinking machines allow this to happen?
Corporate brainwashing, why? That is just realistic. I mean we know earlier people with much harder lifes actually had more free leisure time.. and even Ford imagined with all the automation we may be able to work much less and have better lifes.. still here we are: A few people making tons of money, some soing very good to okayish, but the vast majority doing 2-3 low paying crap jobs to survive.. and we all even workong more than decades ago. How?
plus, before we automate anything civilian, we will have to automate everything military, cause they get the first dabs at any emerging tech.
more likely we will see global war between stealthy autonomous robots much earlier, before we automate much on the civilian side
Based on what science? Grinding at an office 9-5, 5 days a week, 50 weeks a year over meaningless pursuits is in fact what is causing mental health problems in the world today. Give people a social safety net, more time to pursue their hobbies, spend time with their families, connect with their communities, and I can assure you we will all be better off.
Mental health is more related to belonging and sense of community. Currently, not having a job is a source of guilt and shame. That has not always been the case. Nurses and educators find their reward mostly in being able to help others.
In other words, I honestly don't think AI ( in its current state at least ) will change much. I will go even as far as to say that I don't see current generation being able to create an appropriate prompt.
I might be a little optimistic here, but having seen how people normally react to 'easier' things kinda confirms it.
If I worry about anything here, is that AI will become THE answer that you will not be allowed to question.
edit: clarified blackbox statement
We're far from there with AI yet but we're on a trajectory and that's what worries me.
But in the short term I definitely agree with you.
People are already arguing with a bot about why there app was removed from the app store.
Somewhat related, the question might become: do I have a bot that can win an argument with an other bot? As the internet already get flooded with AI-generated seo-spam, we already get to depend on software that outsmart these AIs.
Some people here dream about becoming super productive. Reality might be that we will drown in AI-generated content (code, e-mails, web sites) that some poor people have to judge, clean up and curate.
This is just silly on 2 levels. First is that programming is fun due to the artistic/creative nature of it. It's not what I program that matters, it's how I program it. No way an AI will replace the fun of thinking about code and then materializing that vision.
Second is that once an AI is good enough to write software better than us, IE. rewrite itself better, then we have reached a form of the singularity and all bets are off.
I think the short-medium term future is not that you have nothing to do in a world of abundance, but rather that you are a manual laborer instead of a programmer, lawyer, artist, etc. The future is that you work as a Door Dasher for an automated company and enjoy AI generated art as you do. The car mostly drives itself while you listen to bespoke generated music or podcasts, occasionally taking over for the car and mainly doing "last few feet" delivery - dropping packages and bags off at the door.
For that matter, why do I bother cooking when I could get a better version from a restaurant for just a little bit more money?
I understand this concern, but I don’t think the joy of doing things actually goes away for most people just because we could “cheat” by having someone/something do it better for us.
A short introduction to the culture of The culture can be found here, written by the author himself: http://www.vavatch.co.uk/books/banks/cultnote.htm
Yes, the Minds can do everything but (pan-)humans still enjoys doing things for their own pleasure. Like for example learning to play an extremely difficult instrument while a Mind avatar taunts him by perfectly playing the same instrument. The Player of Games still plays games, although he couldn't ever win against a Mind.
And, should this be not enough, one can always leave The Culture and try and find meaning in the short, brutish lives of primitives people.
InThe Player of Games, the inciting incident for the main plot is that the titular character cheats, accepting help from an AI to maintain his reputation of ludic brilliance against an emerging rival who exhibits greater natural talent.
Subsequently, he is gently blackmailed into participating in a game on a distant planet where life is nasty, brutish, and short to an extreme degree, but where he is accepted as a guest because of his great reputation. The Minds have chosen this method to destabilize the current balance of power and bring the planet in question out of a particularly repugnant developmental minimum. It's ultimately revealed that even the initial moral misstep was not exploited but rather engineered by the Minds from the outset, and that the great player was never more than a pawn in their own games.
Second thing is, since this is a hacker forum, most of us are in a field where supply exceeds demand. So we are very comfortable with the thought of destroying our own business, because we don't truly grasp the reality behind it. Because we are the elite right now who have destroyed older businesses through software decades ago. Who's to guarantee that AI will not result in a rapidly shrinking centralized elite that does not include you? There is no guarantee of AI being shared equally amongst all in the post-scarcity fantasy.
"The Unabomber Manifesto will shape the 21st century the way the Communist Manifesto shaped the 20th. I don’t agree with the conclusions in either, but they state the problem well." -- George Hotz
Spoken very confidently and purely anecdotally. Most of my hobbies I pursued in my formative years as a child, I was neither cajoled into them by my parents and they certainly weren't a means of an escape from school.
If anything I had more time as a child - summers were completely free and lackadaisical, and I performed those hobbies just the same.
If your underlying reason to pursue something is because it's difficult, then I would argue your motivations are flawed. I chose to play the piano as a child because I love the sound that I could produce, being able to glide up and down in sweeping arpeggios made me happy. It's a simple as that.
I will specify my stance on this for childhood since it is a special case and the brain is also very different. So: I also had hobbies as a child that I pursued for their own sake. The key thing is when you grow up, you are either able to pursue those hobbies, or not. You can only do the former if they provide enough value to the world in that case it may become a hard thing, latter is as a catharsis.
> If your underlying reason to pursue something is because it's difficult, then I would argue your motivations are flawed
That’s separate from hobbies. I’m saying we need hard things in general to grow. They come to us, not that we seek them like hobbies, although sometimes it may overlap. For example what was the one thing you didn’t have as a kid? Or something you yearned for, and still didn’t have as an adult? I really don’t think people who “have it all” are that enviable. It has to be a balance. When you go through a process for that, you grow. It’s the “chase” or the “journey”. Very different from leisure time. Both are important.
We are shifting that balance now, and shifting it on either side is not good. I would say we have had enough technology for a utopia for the last few decades. The problem that remains is political not technical. AI will only exacerbate that technical problem, without solving the incentive problem. See social media designed to make you stay hooked. Will the next generation want to play piano or draw stuff when they are in a forever trance of amazing content delivered by Big AI?
The question is, knowing this, will we be able to simply enjoy the satisfaction of our ultimate goals—- endless consumption of automatically generated art, food, sex, drugs, love, etc. Or will we feel forever hollow in our failure to accomplish ‘genuine’ instrumental goals, in a way that cannot be overcome by the ersatz instrumental goals of video games?
Perhaps ultimately we will create virtual environments in which we are perfectly deceived as to their virtual nature, so as to experience the satisfaction of ‘genuine’ instrumental goals, and in so doing come full circle.
I think that not all human desires are a matter of consumption. Some wise people say they even got rich by giving. At least love is such a thing.
My stance is that art and love cannot be consumed, it is what happens to one. Now don't ask me to quantify or proof that. I will leave that to a future AI, but for that it needs to be a perfect, transcendental mind.
It's about turning a specialized craft that people can make a living from into a proscripted commodity task that you can pay slave wages for.
This isn't new. It happened in farming, clothing and food preparation and it's coming for trucking, programming and everything else that pays well.
The project is one of collective enforced impoverishment by substituting labor for property.
Market forces drive innovation to making all human effort worthless and disposable
The reality is engineers need money but they prefer manipulating machines to people, so the business people take care of the unpleasant wetware programming and not coincidentally take the bulk of the profits.
it'd be far easier for a smaller group of engineers to build an AI powered CEO/C-suite that generates the regulatory filings etc. but does all human interaction over zoom or by phone. This would require a bit of work to pull off and would probably be denounced as horribly illegal, but I'm not convinced it would do a worse job than the median C-suite.
We've proactively organized our economy to produce these kinds of hostile outcomes. It's not the technology that's the problem, it's the unquestioned assumptions of how we've collectively presumed it will be used.
We could build things for the collective benefit of humanity but that concept is extraordinarily foreign to us. We've become all Hayek, all the way down and these are the consequences; where all forms of progress can only be imagined as new forms of abuse and enslavement.
That's how it fuels reactionary conservatism. Everything is privatized so these exciting scientific breakthroughs can only be seen through a lens of hierarchy and property and the autocratic despotism that comes with that.
We could break that cycle any time...
Yes, our imagination is bound by the belief systems we are trapped in. We choose possession of plastic widgets with built-in planned obsolescence over giving poor people a cancer treatment.
It is collective behavior driven by belief. Public access to education would benefit all of us, but we don't do that because of some doctrines. Point is that what we leave unquestioned by believing it is just rational, blinds us.
When we will start to rationally and empathically examine our collective memes, we might allow ourselves a better future.
It's more likely that non-technical people will be pushed out of jobs near software in favor of (former) engineers. Someone non-technical writing prompts for code can't actually read/debug/fix/deploy/integrate the result. Someone who is technical can probably write better AI prompts that yield something usable than someone who is non-technical. Plus they'll know how to handle the result.
The predictions about non-technical roles firing all of the engineers and thinking AI will write all of the code don't really hold water for me. We might see an overall workforce reduction, but engineers will probably be the last ones to leave.
I can’t imagine a world where they fire the devs but still need a HR person to email everyone about enrolling in health plan or whatever.
It reminds me of NIMBYism. We’ve been automating entire professions for over a century now… but not MY profession…
If you're not valuable to the economic system you won't be treated well.
>It reminds me of NIMBYism. We’ve been automating entire professions for over a century now… but not MY profession…
Yes... it's self interest look at doctors or unions or guilds.
If we don’t assume that, then there is never any useful conversation possible on this topic. We end up with the ridiculous “we need jobs because that’s what we do!”
Which leads to another NIMBYism that I see when this conversation is had: “some generations will have to suffer through the friction of an economic revolution but not MY generation.” I think we need to be prepared that there’s always a chance that we get to be one of those generations.
You’re right that there’s a self-interest there. It makes it almost a good thing that engineers are far too interested in the means rather than the ends.
If the holders of capital who are best positioned to reap nearly all the benefits of automation aren't willing work towards some more equitable result why would I want to help automation at all. I'd rather work against it.
"just ignore the rising inequality and complete collapse in value of human labour we can work out the details later once I hold all the cards" is not a compelling story.
The bleakness you feel might be due to a poverty of imagination.
> Life will be like a computer game with cheat-codes, whenever we struggle with something, instead of fighting on and improving we will turn to our universal cheat-engine: AI.
One such example of the above.
Thank you for your diagnosis. It may _also_ be due to the opposite. If you know anything about how the brain works, you'd know that in general more imaginative people are more anxious.
Peter Norvig: This is why AlphaCode learned to write code with one-letter variable names, and with no comments or docstrings.
!!
That is to say, golfing is bad unless for fun, shorter code is usually better, the shitty hard to read code is probably a reflection of human laziness rather than misguided tersity.
I might be terribly wrong though. :-D
Typography, layout, and presentation counts for more than coders such as myself will ever admit publicly :)
This has nothing to do with the parent comment's topic, which was about using computational notebooks as blog posts, and is in effect "complaining about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage" - which the guidelines explicitly forbid.
1. Write a static file server in Go
2. Write Go code to convert Color image to B/W
For both I got results. I know both are simple but still it's fascinating that AIs can write code. I have written more about it here https://rohanrd.xyz/posts/surprising-capability-of-ai-code-g...
Afterwards I asked it to add an option to "deep fry" the gif. Not only did it produce the correct code it also understood what I meant when referring to deep fried gifs. I was definitely impressed.
I haven't tried anything more advanced, but to go from simple requirements to solution without any clarification or even method signatures (It guessed the correct method signature down to the name and input params) was pretty dang impressive to me.
I think using more diverse and high-quality training data could help address these issues, and incorporating additional constraints and regularization techniques into the model's training could prevent hallucinations and improve its overall reasoning abilities.
While there is room for improvement, the progress in this field is exciting and I can't wait to see where it will lead.
NB: This comment was written by GPT-3 after reading the article. The last few months of AI have been frankly mind-boggling.
If a coder can write code, but they can’t do that, they’re useless to the org. It will be faster for me to write their code myself than to maintain what they’ve done.
So really that’s what I’d need from an AI coder. Writing the code is good, but can we talk about it and can you learn the specific architecture principles we have applie in this specific codebase.
Therefore, in most orgs, code architecture/style is a distant secondary to 'does it achieve what the user will pay money for' and 'why isn't it finished yesterday?'.
The point is to stimulate brainstorming, not get answers.
IMO it's more like a coder using AI to make their job easier. It's still up to a human to come up with the individual problem the function solves, architect a solution from multiple functions/objects/etc, come up with a data model, and so on and so on. The AI just generates the code itself. And at least as of now, the code needs to be double-checked.
Sure, there will always be demand for a Linus Torvalds or a Damien Hirst. But will there be demand for Coder #365968 at Infosys or a graphic designer pumping out $50 ad banners?
We’re looking at the possibility of some white collar jobs having the same income disparity as creative jobs. Just as there are some musicians who make hundreds of millions while the vast majority barely make ends meet, we may have a future where the star programmers make millions while the average players are automated out of the competition.
Eventually, I agree yes. But there could be a boom of huge new investments into A.I products, more devs needed and in fact teams getting way more requirements since they are more productive. Imagine the stuff we will be able to build in things like search, personal assistants, biomed, in fact what industry won't this affect? Its unbelievable to me that people are now saying Google search might become obsolete, that's absolutely crazy. Not many people saw that one coming. But at least initially I don't think GPT models will be able to do everything themselves. So its very hard to determine that say in the coming 5 years devs will find it more difficult to get a job. 10-20 years from now sure, I don't see how anyone gets a cognitive job anymore let alone devs. In fact our entire school/university system is probably obsolete, kids are probably learning skills they won't be able to apply in any job market. We need to start think about stuff like teaching kids emotional intelligence, spirituality and meditation...not cramming for a math test.
If an ad agency or a magazine publisher can get a custom illustration that works for my purposes from Dall-E, then they ain't paying no artist. Not theoritical, many already do use those generated images.
That's not just "making the artist's job easier". It's taking jobs from artists (well, illustrators and graphic designers at least), especially in the cheaper end of the business (e.g. not Nike, but your local Pet Store chain, restaurant, or news outlet, sure).
So yes, not OP, but I think artists will still have jobs, although less of them, and the job will be different.
Midjourney V4 is amazing. It spits out absolutely beautiful images.
There's enough art talent already around in the world to entirely commoditize the supply of it for the little one-off no-style-guide-to-follow commissioned works you're talking about. It's just not currently a liquid market — supply and demand find it hard to discover one-another — and so a true market-clearing price can't be set.
Meanwhile, AI is not currently taking anyone's advertising-campaign graphic design job, or anything else where the "efficient-market price" (in a world where human "art sweatshops" existed) would be more than $5.
I find it best to accept it will most likely replace all of us, from doctors to coders to even psychotherapists. Won't happen next year but 10-20 years is a very long time this thing keeps getting better. Eventually we won't be able to tell if its a machine or a brilliant superhuman. The bummer in all of this, in my view, isn't the loss of jobs; we'll find what to do. It's the transition period - the accounting wizard or the brilliant doctor losing their jobs and status and becoming kindergarten teachers or care takers or unemployed. Nothing in their upbringing or life experience prepared them for such a thing ... so that's probably gonna be rough for many people. But once most people went through the transition it won't be bad. Society I believe will be better off. We will stop being obsessed with money and status and spend much more time with family and friends. Entertainment will be insanely good and so will healthcare. Possibly medicine to make our moods better. It could be utopia.
Yes, this is a big problem.
However... a lot of people would enjoy being teachers, albeit with significant improvements to the educational systems.
I doubt very much that this is a testable theory. I think it is primarily a normative one.
This is a prediction?
Given human nature and the diversity of people (w.r.t. rationality, religiosity, morality, capability, and so on), it is very much an open question about (a) how AI capabilities will develop; (b) how they will be paid for... (c) and by whom; (d) to whom will benefits accrue; (e) how will society change.
These are broad, sweeping questions. Plenty of fodder for imagination, hope, transformation, cynicism, backsliding, or even despair.
If I were to make a bet, on our current trajectory, I see some key factors in tension:
1. educational quality, in absolute terms, increasing _and_ being more equitable
2. educational quality, in relative terms, continuing to be very unequal and probably getting more so. As one example, who has the resources to direct computationally intensive AI experiments? There are (and probably will be for a long time) gatekeepers for these resources. People that mix in this circles have a huge advantage. This makes me wonder if "exclusivity leads to inequality" is a saying from some philosopher.
First, I don't see "utopia" as likely; furthermore, I have a suspicion it may be impossible, given human nature.
Second, even the argument that society will be "better" demands much more reflection. The implied argument above is only a sketch. I don't find it convincing much less plausible. I'll call attention to four points (implied from above):
1. AI will replace humans in most or all professions
2. AI quality will be much higher than the previous human levels
3. A broad swath of people (using some notion of equity and fairness) will have enough money to live happily
4. "We'll spend much more time with family and friends"
Each of the four points are quite uncertain. Furthermore, even if `k` is true, `k+1` does not follow.
Who would like to flesh out some ways the sequence (1, 2, 3, 4) might happen?
Anyway speculating is fun but you're right its just speculating. My main point is we should always keep in mind this could turn out to be great .
_And_ if since we care about our AI-interdependent future, more of us (as in the people here on HN) need to wade into the gory details, including ethics and the current power structures. The "technology" (as in algorithms, data structures, hardware, etc) is arguably the "easy" part. There are plenty of existing incentives and structures to keep those _moving_. But moving in what direction? Even the notion of an "ethical compass" seems antiquated in light of current technology. We may have to reframe everything. This is a big challenge.
They get nothing but tents and shame.
Where are the examples of the middle voting against the extremes when times get really hard?
The Star Trek post-scarcity utopia scenario feels very unlikely; Mad Max-style scrapping for leftovers while Musk, Bezos, et. al. live behind walls feels infinitely more probable.
How do you implement UBI when a huge proportion of the political class is vehemently against it. Maybe we need to AI politicians, so they start to figure it out?
And remember, we are still a democracy. We get to vote. We control the army and the police and all institutions. If we decide that this capitalism isn't hot sh* anymore we can change it. What will the evil billionaires do? (this sounds like a good straight to DVD movie actually...hey GPT write me a script about this)
As nice as this would be, I think there's roughly 0% chance of it happening. Over the past few millenia humans have doubled productivity per capita a ridiculous number of times. None of those leaps led to an end of status seeking or a transition towards mostly leasure time for the masses.
Instead I expect more of the opposite from these developments. Power will get increasingly concentrated with people who have very little interest in the needs and wants of the plebs.
At least in OpenAI's case I think they take this thing very seriously (Sam Altman doesn't strike me as evil one bit, quite the opposite in fact https://www.youtube.com/watch?v=DEvbDq6BOVM). In fact most of the tech elites don't seem evil to me, if they only cared about money Zuckerberg and Gates wouldn't have pledged away all their wealth. Some of them are as you describe but I think most of them are actually somewhere in the progressive axis.
So far it hasn't worked out too well...
It's up to us. Indeed easier said than done in the current dysfunctional and polarized politics of ours but we can still do it.
For the philanthropy, I'll just note that these pledges don't involve literally transferring 99% of wealth out of their control. The vast majority of the pledged money goes to a trust the person controls, organizations they have some relation to, or just stay completely in their control for years with only vague non-binding commitments to eventually donate it. In return for this largess they get significant reputational and tax compensation.
If you don't understand the program ChatGPT wrote, it will happily butcher it for you, because it doesn't really understand it either.
Don't worry, you wont be there at the org to make these "fruitful conversations" either. The AI will take your job too
I've been waiting for something to take my job for 15 years.
I started as a small developer testing radio firmware, moved on to test web firmware, now I instruct terraform how to build infrastructure and now I instruct developers on how to do things to build proper infrastructure.
I'm ready to retire but apparently I'm incapable of having an AI that can actually simulate Super Power ADHD at work, so we'll have to wait a bit.
Every company has suffered from the decision, in my experience. Never has it destroyed the employer, but I have heard stories about such eventualities.
For all the important contributions Steve Jobs made to Apple when he returned, maybe the least heralded and hardest to implement is that he managed to NOT make himself irreplaceable. To many people's surprise, Apple did not collapse after his retirement and death. So maybe his most genius contribution was not to make Apple dependent on his ongoing genius contributions.
> So maybe his most genius contribution was not to make Apple dependent on his ongoing genius contributions.
That's because its even lower level characters in managements deciding...
This wont fly at organizations that need the code to work, every time, but think about the explosion of non-programmers who can now make systems that “basically work”. If you don’t think “basically works” is a high enough bar to succeed in e-commerce, let me show you my recent support email threads with companies from whom I’ve been trying to purchase Xmas gifts for my wife.
But this thing is not a sub for coders. It is an assistant to coders. And yes, being able to explain the code is incredibly important. Just as it is to verify the code.
To me, this is just another exercise where coder becomes manager of his very own coder. And has to check the code his coder produced.
You feed it a large input corpus and it digests it in clever ways. Then when you probe for something contained wholly within the space described by that corpus, it is amazingly good at fabricating something plausible to match the point in that space that you requested.
Which covers a lot of stuff and is very useful, but does very little for problems that require extrapolation. It can't expand the edges, it can't come up with anything truly original. It can't solve problems that people haven't already solved and written down the solutions somewhere the AI could find them.
Another way of saying it: AI today is much better at memory than thought.
Career advice for young people today: specialize in pushing the edges, not in filling in details. There will be some areas where applying existing stuff will hold out and be useful for a long time, but you'll always be racing against the AI. Colonize the parts of problem space where AI doesn't have the imagination to go.
(Of course, humans are notoriously bad at correctly recognizing what does or doesn't require originality. Hell, we think we're making decisions about what to do every minute of every day, when in fact we're just a bunch of dancing meat automata following ingrained patterns 99% of the time.)
There is a process that generates ideas/solutions, and a process that tests them. An artist and a critic, a scientist and a lab. Together they form the experimentation loop.
Let's take the game of Go for example. Testing who won a game is trivial. AlphaGo managed to beat humans in a few days of self-training. In other words, those edges you speak about can be pushed with massive search and verification.
There is no reason we can't do massive search + verification for math and code. This is a good way to create training data where it doesn't exist in sufficient quantity.
Other things can be simulated with expensive computation, and then "distilled" into fast neural networks. Then we apply the neural net to fast-search solutions. In the end we need to verify some of them (thinking of weather simulations, new materials, new drugs, ...)
Also reminded of the recent AlphaTensor who leveraged massive learning from verification to beat Strassen's algorithm who was state of the art for 50 years. The is no reason neural nets should remain purely interpolative if we can manufacture good training data by running computation or experiments.
Ideas are cheap, verification matters. Generative outputs are worthless without verification.
Could you please provide some support for your argument? This was repeated a lot in the early days of the modern wave (2012-2016) but was pretty thoroughly debunked as we've explored generalization and how these models disentangle intrinsic concepts and compute with them. Heck, even modern transformers are restricted memory Turing complete and use that to their advantage.
Also, I do not mean to be rude, but frankly saying "colonize the parts of the problem space where AI doesn't have the imagination to go" is frankly rather terrible as it leans on the imagination argument of AI. At this point, being adaptable to co-integrate will be good, otherwise I could see people following that stuck inside of some kind of Sisyphean pseuso-Luddite escapist nightmare.
Source for opinions: have been involved in ML in some form for most of the modern wave, and am appropriately (quite) skeptical about the AI takeover/revolution/eventual singularity belief/etc.
It cobbles together examples, sure, but that's exactly what we do too. Everything "original" we make is riddled with subconscious outside influences.
That said, I think it’s still fair to say that most interpolation type of work is looking very threatened by AI as it is today, so the converse (don’t go into those fields) is probably sound advice in light of current developments. As for the rest, I suspect we’ll be forced to chisel off piece by piece from our zeitgeist of “imagination”. Some pieces will fall off quickly, as AIs replace them, and some will take longer, perhaps a lot longer.
I do find it interesting that computers ended up killing it in unpredictable domains such as style transfer and NLP, while being mediocre-at-best in eg humor. It may mean that we have over- and underestimated aspects of what traits are unique and sophisticated.
You're dead wrong about that. Remember Deepmind's original claim to fame? Alphago?
It came up with tactics and strategies that top professional go players considered to be novel to the point of being revolutionary. If you look at the way games are played by human professionals in the years before and after Alphago came on to the stage, you'll find they're different to an extent greater than any other upheaval in the many centuries-long history of the game.
AI is a much more diverse field than you seem to give it credit for. Paint with overly broad strokes like that, and you're bound to go astray.
In essence there is no such thing as extrapolation for the human mind. We only interpolate in new ways. We can only remix what we know, and with that generate something new.
How do we actually get new, non-interpolated stuff? By going out into the world and doing things with it. Collecting new data. Experiments, experiencing things, seeing new stuff, reading about new stuff.
That's not so different from the AI.
Our brain is not something magical that works in mysterious un-replicable ways. Much of AI field has been in fact trying to replicate how we learn, and how our brain works.
Building the thing right vs. building the right thing.
I am not saying that AI will not be able to revolutionize a lot of areas, but solving well known coding competitions are far removed from where a large portion of coders and technical people make their money.
A software engineer's job will be the same as it always was – to translate unclear and always-changing requirements into something that works. Until ChatGPT, Deepmind or whoever else can learn to deal with my client or product manager, my job is safe.
In fact AI making programmers more productive is a good thing. It is only going to increase the problem space and we'll need yet more engineers to fill it, like all other productivity advancements that came before.
You have to see this from the perspective of non-tech companies, who see tech as a cost-center, not a profit or innovation center.
Does your regional grocery store that just needs some tools that can help it track inventory really care whether its code comes from a team of big brained humans or two programmers in a basement copy-pasting chatGPT answers?
A single programmer with a laptop can do in minutes today what it took entire companies of hundreds of professionals and a large amount of funding a few decades ago. Yet the size of the industry hasn't shrunk in the same period – quite the opposite in fact. There is more need and demand for programmers today than ever before.
It's going to be an exciting few years to say the least.
If cost vs revenue allows for more developers, yes, you can see the happy path. No wonder you observed this at the start of the software revolution. The app stores are already overcrowded.
Given the rise of "NoCode", perhaps clients & managers are more willing to meet in the middle than one would otherwise assume.
Given how managers seem to enjoy meetings, I can easily imagine one sitting down with e.g. Dragon to speak with e.g. ChatGPT in order to clarify, disambiguate, and expand requirements list(s) before feeding them to e.g. Alpha Code.
I do, however, think there will be an inflection point in the coming decades, where the tools become more generalized and better at dealing with new problems. I might also add that the reason for this belief is simply that a lot of work is being put into making these type of generalized tools; but unlike Kurzweil, I don't quite believe it will lead to the Singularity. :)
The loom and sewing machine didn't eliminate the seamstress, it just impoverished the profession.
It's like driverless trucks; what you'll actually see is mostly driverless outsourced and remotely monitored trucks where someone is making something like $1/day monitoring 5 trucks at once and switching it to remote control mode when needed.
We've proactively organized our economy to produce these kinds of outcomes. It's not the technology that's the problem, it's the unquestioned assumptions of how we've collectively presumed it will be used.
E.g, Lai–Yang algorithm will give you no hits,
Fascination observation, I asked for Project Euler #193 (a problem I solved using mobius function). chatGPT solved it using bruteforce, even after I asked for the most efficient way it could solve it with. It just used memorization, which wasnt enough to find an answer for that problem quick enough. I asked whether it could use the mobius function, it couldnt translate it to code, and I had to give it python code to make it work.
- If you ask in-depth technical questions on how task queueing works within Elasticsearch, it wont be able to give you answer.
- George Hotz in Lex friedmen interview mentions that GPT-3 has 100 most recent messages as a limit. He isnt convinced that is enough to completely build a complex tool like Solve FSD or implement me a kafka.
-Euler #193 - This problem is not super obvious and I see discussions on it invoking Riemann zeta. ChatGPT was able to create a correct Mobius function on its first try. I have no idea how this might be related to the problem though.
-Elasticsearch - Why on earth would you expect it to know this?
You've cherry picked some super niche stuff that this brand new AI can't do and name dropped some fancy theorems. We're all very impressed.
Actual text: "AI just trounced roughly 50 percent of human coders"
I think we already all knew that about half of the coders out there weren't all that great.
"Think of how stupid the average person is, and realize half of them are stupider than that."
“Oh big deal”
Now they never lose to any human in any chess game ever.
There is a big question though of how this translates to real world programming. Competitive programming and real world work are not the same thing.
https://www.deepmind.com/blog/competitive-programming-with-a...
which walks through an example
It's a lot like the conversation not too long ago that AI was going to replace managers. Sure, some aspect of their work might be gone, but it didn't obviate the need for the human.
[0] https://www.deepmind.com/blog/competitive-programming-with-a...
[0]: https://www.deepmind.com/blog/competitive-programming-with-a...
In fact, it was published a few months before other important papers. And yet, it seems pretty much all the press is still discussing and analyzing the old paper. Why is that?
I’d like to think it’s because the earlier work is so compelling, but more likely someone needs to drive more traffic to their site.
I think this is more of a problem for code ai discussion. This is something you can’t really discuss on a forum or blog post — your posts will get overshadowed by ML doomstering.
Sure, a sufficiently advanced AI can craft a UI that looks like something incredible (e.g. McMaster.com), but it won't necessarily be able to replicate the quality of their customer service, consistency of offerings, or reliability of delivery (the UX).
Small nuances in some domain can cause vast changes in how you would be expected to interact with it. AI typically fails to extrapolate and manage all of these possibilities. A good human developer can play devil's advocate for 30 minutes and come up with edges that would decimate any AI on the horizon. But, take that same pile of edges and hand them to a product wizard, and that person will give you all kinds of workarounds that would be acceptable to the end customer. Kick it back over the wall to the developers and you have happy customers.
I don't think it will last forever, but the most complex domains seem to be the safest ones for a human developer's career prospects.
I feel like this "only" is erroneous. While FAANG do have solid diversity policies in place to assume they're the only companies capable diversity in their hiring practices is offensive to everyone else.
HN has the best thread for discussing the caste baste discrimination that happens among Indian tech workers in the USA. It turns out that being white still gives you a lot of privilege, even in tye FAANGs.
The job is much more architecture/design, creating API contracts, understanding the health of systems in many different ways, measuring impact of changes, etc. Regular engineering stuff at big scale. Obviously there’s plenty of coding too, but it’s not really the important part of the job, and already has a ton of automation for boilerplate.
I imagine that the automation will just improve another step change, there will be more need for review and guidance of the AI algorithms and the engineers will do more of this. And interviews will involve in some way.
That said, its likely that the job of being a dev is going to be largely debugging generated code in the future no matter where you work.
I think this is an important point.
> That said, its likely that the job of being a dev is going to be largely debugging generated code in the future no matter where you work.
This and formulating the right prompts, i think.
I'm a FAANG interviewer and I've run my coding questions through ChatGPT.
The TL;DR is that code would easily pass a junior-level interview, and maybe a mid-level one. I'm definitely convinced these technologies will disrupt leetcode-style coding interviews.
So either we embrace it, or ditch this approach altogether.
I imagine an AI in that kind of environment inventing Skynet and doing us all in.
Isn't this just over-fitting the model?
"As a sanity check and to narrow the results down, the AI runs candidate solves through simple test cases. It then clusters similar ones so it nails down just one from each cluster to submit to the challenge."
The big question for me is, where did the test cases come from? They seem to be contributing much more to the outcome here than just "a sanity check and to narrow the results down." If they were effectively spelled out as part of the challenge, then this is an impressive result extending the trajectory of other recent achievements, but not so far as to justify the title's "conquers coding." If they were not, then this would seem to be taking things to a whole new level.
This was essentially a competition in the speed of programming. But if we want to discuss practical application, as in laying off armies of coders, we need to realise that there is a tremendous gap in the productivity of, let's say a solo startup founder and the productivity of a team of coders working for a multinational behemoth barely producing anything of a value over the whole sprint.
Gave me a good chuckle lol
> Write an algorithm that, when given a sorted list of numbers, efficiently finds all numbers in the list that lie between a given a range
And this is its response:
> Here is an algorithm that can be used to find all numbers in a sorted list that lie between a given range:
Initialize two variables, low and high, to the lower and upper bounds of the range, respectively.
Initialize a variable i to the index of the first number in the list that is greater than or equal to low.
Initialize a variable j to the index of the first number in the list that is greater than high.
Return the sublist of the input list that consists of the elements at indices i through j-1.
Here is some example code in Python that implements this algorithm:
def find_range(numbers, low, high):
i = 0
while i < len(numbers) and numbers[i] < low:
i += 1
j = i
while j < len(numbers) and numbers[j] <= high:
j += 1
return numbers[i:j]
Does this algorithm really have a time complexity of O(1)? I would think it is O(N). Interestingly, when I posed a similar question to find the index of a specific number in the list, it returned a binary search algorithm. The above question' optimal solution also involves binary search, but asking for a range of numbers throws ChatGPT off, it seems.
If you work at a place where devs are just code monkeys who implement the orders from on high with no feedback whatsoever or avenue for pushback... get a better job.
How would you swap two integers without using a temporary variable? Seen it before? Pass. Not seen it before? Fail.
Steps, mostly driven by just basically knowing the goal and that there's not many operations that could possibly help:
"a" "b"
"a+b" "b"
"a+b" "-a"
"b" "-a"
"b" "a"
Then once you have that, you can enumerate the downsides to that, look for more efficient and less error-prone ways to proceed.
We were looking for the solution based on XOR which works with only two registers.
We wish you luck with your job search elsewhere.
These interview questions are fairly shit, but they're not _that_ shit, and the people giving them are doing their best.
In nearly 30 years of diverse coding experience, I've never once encountered a situation where this solution would be useful.
That said, its probably more used as a filter for people interested enough in working there to study.
To put it another way - they can already automate solving their leetcode problems by looking up the solution in their database, no need for AI. But that's not the point at all.
If they aren't evaluating how the code you write fits into the context of a real world problem, how can they possibly use it to asses your suitability for the job? Using fake code problems to evaluate candidates is the flawed premise here.
Like I said, they were always able to automate solving those problems by doing a database lookup.
You are contradicting your earlier statements. But yes, a tool that can beat leetcode interviews successfully and reliably is a game changer for candidates who want to make it through these pointless LC interviews as a whole.
Because there was a time we did this, and with tools like Google Search it became dumb, but it took some dinosaurs a really long time to let go of their old ways. Hell, some of them are probably still around.
At the end of the day, aren't ChatGPT, AlphaCode, etc still just finetuned LLMs, or has there been some significant changes to the architecture that give reason to expect the output to be anything more intelligent than LLM "predict next word" continuations ?
Of course GPT, Codex/Copilot, ChatGPT are all impressive, and have some unexpected emergent capabilities (this being the interesting thing about this "scale experiment"), but even with "think step by step" prompt hacks and finetuning, they're still just LLMs unless something (intelligence!) has been added so they can learn from their mistakes at runtime and test their predictions ("understanding") by interaction with the external world.
So, if these are still just LLMs, then why are people treating them as if their prompt answers might/should actually be correct as opposed to what might be expected from massive training set prediction - i.e. right a lot of the time, but catastrophically wrong a lot of the time too, with no insight as to which is which.
I saw a ChatGPT example the other day where it confidently explained how taking 90% of a number then adding 10% of that would get you back to the original. It sure doesn't appear if there's any intelligence there, even if it seems it's generating "correct" answers enough of the time that people want to ascribe intelligence to it. I'd trust advice from ChatGPT about as much as I'd trust advice from Eliza (a simplistic 1960's ChatBot that also managed to fool some people).
I'm not sure what it says that this type of "contextually correct bullshit" is being productized into things like OpenAI Codex. What next? Use it for medical advice too ? If I want untrustworthy suggestions I'd rather just Google for it where I can at least judge the source, as opposed to ChatGPT where you have no idea whether it's regurgitating stuff from NASA or 4chan.
RPG games will include options for laws of karma or configuring an omniscient AI "god" who ingests all of your actions and develops consequences or new plots and dialogue. And of course the player can pray to the god for forgiveness, for blessings, etc.
I feel dizzy trying to think through the social and emotional ramifications of becoming seriously attached to NPCs who seem real and can articulate their feelings and share experiences with us.
In the real world (outside of programming competitions, that is), programmers don't get clear instructions for what we're doing, and what instructions we get deal with messy aspects of the real world. For just one example I tried: if you ask Codex how to calculate taxes on US products, it misses the point of the ask. I know that you have a tax rate and a price and you multiple them together - but what tax rate is applicable? Figuring that out is a thorny problem.
In programming, a lot of the inherent complexity is in understanding the weird bits of ambiguity and the corner cases. Whether that specification is through natural language or some other thing, we're a ways of from a computer capable of replacing us on that.
I suspect it would not do a very good job at writing test code for itself. Until it can prove that it writes code that works, it’s only really guessing.
Are we going to end up with AI training on AI generated datasets.
Some AI speculators, using Gödelesque reasoning, posit that computer programming will be the last job to be automated by AI, so it can bootstrap itself. In reality, it will be one of the first jobs automated by AI, while jobs like cook, barista, etc... will be much later automated, if ever.
>
> Here AlphaCode could not come up with an algorithm to solve the problem [snip]
>
> But then AlphaCode behaves a bit like a desperate human, and hardcodes the answer for the example case to pass it even though its solution is wrong, hoping that it works in all other cases but just not on the example. Humans do this as well, and such hope is almost always wrong - as it is in this case.
It behaves just like me...
This isn't the same question though. The ML models that were trying to beat Go were on the correct path, it was just a matter of improvement. It was inevitable that they would beat the game.
ChatGPT doesn't logically reason about code or business contexts. The developers of ChatGPT are not trying to do that, the developers don't know HOW to do that, not even slightly.
Until they're on that pathway it's not even a question: ML that very efficiently copies and pastes from StackOverflow is simply not going to significantly replace programming jobs. The question of how long is left until the AI replaces programmers is completely irrelevant right now.
Using Go/Chess analogy:
> If the game is to replace effective human programmers then the AI is not even playing the game yet, let alone X years away from beating it.
I'm sure every programmer has their finger crossed that the long tail is very long. And not that I think programmers will be out of jobs, probably not, but their value as a prompt technician will be substantially less than as a software engineer.
I've never felt that way about any tool.
Everything that ChatGPT does can be done manually. There are weak programmers who essentially piece things together from StackOverflow and other sources all day, and just brute force "make it work". Did the emergence of StackOverflow replace programming jobs? No way, there are actually far more programming jobs now then there were when StackOverflow was founded. The reason is simple:
> If you want to create complex production software and grow and maintain it long term, it's simply not enough to copy and paste from the internet.
So therefore a machine that does exactly the same thing can't either. Go and build a complex ERP system using only ChatGPT answer, just try it. It will be a steaming pile of shit, riddled with technical debt. You won't be able to build a good business from the result.
Right now even if you use ChatGPT you inevitably need to be a developer to fix it's errors, so it's currently replacing exactly ZERO developers.
When we have AI that can logically reason about code, along with the business context it's being applied to, then it will significantly replace programming jobs. We're absolutely nowhere near that, ChatGPT doesn't even sniff anywhere near doing that.
Personally, I can't even remember the last time I went on StackOverflow to find answers about something, and there's enough false positives where reading documentation is far preferable to asking ChatGPT about how something works. So currently for me ChatGPT doesn't save any time, I do use Copilot though.
Language models have been very good at spitting out chunks of text verbatim that were in their training data. Buried in the github training data will belots of examples where people post their solutions to fun problems..including past code competitions.
How often is it managing to match a description of a repository and spitting out correct code that cribbed heavily from it? That is, instead of figuring out the problem, it is pattern matching to a solution that someone else already figured out?
How would we know?
[0]: https://www.deepmind.com/blog/competitive-programming-with-a...
I suppose that might count as performing as well as (some) humans.
It's pretty impressive that this naive approach works on something as complicated as coding. In hindsight it seems "obvious" that it would work in game-like contexts like chess/go. Programming feels like its a significantly higher level challenge in a way (not that go is trivial)
Not sure if you could compare it, but it seems that the next gen coders will say to the AI what to code rather than to code by themselves. A complete different skill set.
You still want to have people, who actually understand what a piece of code or program does. A magical black box to throw prompts at, might be nice for simple settings, but potentially can cause major fuck ups for complex systems.
I think "conquers coding" may be overstating it.
Until then, human are needed to build those AIs.
[0]: https://www.kaggle.com/competitions/abstraction-and-reasonin...
> While not yet on the level of humans,
My job is safe I see.
Title: Online Website for Chocolate Donuts
Objective:
The objective of this website is to provide a platform for customers to order and purchase chocolate donuts online. The website will offer a variety of chocolate donut flavors and allow customers to customize their orders by choosing toppings and packaging options.
Functional Requirements:
Customers should be able to create an account and log in to the website. Customers should be able to browse and view the available chocolate donut flavors, toppings, and packaging options. Customers should be able to add chocolate donuts and toppings to their cart and specify the quantity of each item. Customers should be able to view their cart and checkout to place their order. The website should process payment and provide confirmation of the order. Customers should be able to track the status of their order and receive updates on the expected delivery time.
Non-Functional Requirements:
The website should be secure and protect customer information. The website should be easy to use and navigate. The website should be accessible on a range of devices, including desktop computers, laptops, and mobile devices. The website should load quickly and be responsive to user actions. The website should have a clean and professional design.
User Roles and Permissions:
Customers: Customers will be able to create an account, log in, browse and order chocolate donuts, track their order status, and access their account information. Admin: The admin will be responsible for managing the website, including adding and removing products, processing orders, and managing customer information.
User Interface Design:
The website will have a clean and professional design, with a navigation menu at the top of the page and a search bar for customers to easily find specific products. The home page will feature a carousel of images showcasing the available chocolate donut flavors and toppings. The product pages will display images and descriptions of the chocolate donuts, along with options for toppings and packaging. The cart page will display the items in the customer's cart, along with the total price and a checkout button. The checkout page will allow customers to enter their shipping and payment information, review their order, and place their order. The order confirmation page will display the details of the customer's order and provide an estimated delivery time. The account page will allow customers to view and update their account information, including their shipping and payment details.
Database Design:
The website will have a database to store customer information, including their name, email address, shipping and payment details, and order history. The database will also store information on the available chocolate donut flavors, toppings, and packaging options. The database will keep track of the current orders and their status, including the expected delivery time.
Technical Requirements:
The website will be built using HTML, CSS, and JavaScript for the front-end, and a server-side language such as PHP or Ruby for the back-end. The website will use a database management system such as MySQL to store and manage the data. The website will be hosted on a web server and accessed through a domain name. The website will use a secure socket layer (SSL) to encrypt data and protect customer information.
Testing:
The website will be tested to ensure that it meets the functional and non-functional requirements. User acceptance testing will be performed to ensure that the website is easy to use and navigate. The website will be tested for compatibility on different devices
It lacks the human soul.
It's theft.*
It's unethically trained.
As a coder, I'm exploited!
It's going to take my job!
* only if it spits out licenced code blocks and comments verbatim