This Time It's Different
justus.pw
justus.pw
If you're a 14-15 year old computer enthusiast, planning on getting a BS or MS in CS, you'll likely first enter the workforce 7-10 years. The time you spend in University is a lifetime for certain fields of Machine Learning / AI, and who knows what entry jobs have been completely automated.
Personally, I think this will be the death of entry-level jobs.
Perhaps we'll see similar difficulties with these LLMs. They are definitely not quite there yet.
I'm currently trying to get ChatGPT to write some Python code that formats SQL statements according to my idiosyncratic preferences and it keeps getting very close but never gets it exactly correct.
But I am also sad, because I do think the "making useful things with computers" work will (and has) become less focused on the parts of it that are fun for me. I like to step through debuggers and reason from first principles about what's going on in cpus and memory chips and on the network think about how to structure blocks of code to be comprehensible; all the old school "programming" stuff, that's what's fun for me. I get some joy in working with groups of people to identify and build useful things, but honestly not as much as I get in the direct work with code I understand and appreciate. And I do think more and more of the work will shift away from that raw programming work and toward supervising AI tools in creating useful things conceptualized by humans.
First there is going to be massive amounts of money being splashed around making sure everybody has access to an AI that they control. Both big companies, and on our computers at home. 10-20 years at least.
Then we need to actually do the work of automating all the boring stuff like accounting, taxes, insurance etc. (At lets face it, governments are very slow to change and it will be 50 years till we work out the policy let alone the tech implementing the policy)
Meanwhile, the tech folks need to turn the attention of the AI to saving our environment. We need massive scale carbon capture of some kind. We need renewable energy sources. AI is not going to just work it out and build it all itself. We need to tell it what we want.
We need drones to fly around our forests looking after the bunnies and the squirrels, building a balanced ecosystem.
Somebody needs to develop and AI that will monitor our own health, understand more deeply how our bodies work, how to navigate nanobots though our system zapping cancer cells and balancing our gut bacteria. It would be nice if AI could solve aging in the next 20 years. (better to reverse it in my case)
Somebody needs to develop and AI and robots that can feed us. Micromanage greenhouses and livestock. Farming has a long way to go before its sustainable and ethical.
There is massive amounts of work in just building better infrastructure around our cities. How to clean our water and get people from A to B as comfortably, cleanly, and efficiently as possible.
Then once all that is done we can look to the stars, build the arc ships, figure out how much dirt we need to take with us to sustain us.
All the while we need to be entertained with stories, games, music and more!
This had been said about everything AI related, but the evidence shows it is actually the "higher level", jobs which are being disrupted first.
It is a matter of incentives, and the human tendency to assign blame to people. Everyone can blame their manager, but entry-levels are not blamed by anyone, because they are not supervising anyone.
Technology is in the business of replacing in-betweens, middlemen, rent-seekers etc with systems that are less biased, more objective, more transparent (sometimes), less prone to corruption etc. That's why google became more trusted than webrings, and why people use yelp or google maps instead of asking the locals.
Therefore, there are more people who want to replace their supervisors/managers/superiors with AI than the converse, and this will drive more entry-level people to work for companies that offer more automated , more transparent and trustable management processes. Humans are a point of friction
Now, what would happen if all of a sudden a universal Turing machine came along? Well, by virtue of being universal, that means that it can emulate me and all other Turing machines. This time around things are different. Even if I can find a way to incorporate it into my workflow, it can still emulate that more sophisticated version of me by virtue of being universal. So it then comes down to whether or not I can incorporate the latest version of this universal Turing machine faster than its own design is improved. If not, I will be replaced. Since in our instantiation, I am made from biological material it's in my mind only a matter of time before the universal Turing machine starts outpacing me.
So, I guess the question is then if these GPT models (or their descendants) are universal (in my hand wavy definition of the term).
You seem to be confusing UTM with Artificial General Intelligence. Universal Turing Machine is not the term for some magic machine that can interpret and integrate any observed computation. LLMs will significantly change how we interact with computers, but the ability to emulate another turing machine has always been there (for computers and yes, LLMs with memory are turning complete). That doesn't mean AGI can be implemented efficiently or that LLMs are sufficient for AGI.
What were going through now could maybe be likened to what it would be like for a Turing machine to encounter a universal Turing machine for the first time. For all its life this fictitious Turing machine has encountered other non-universal Turing machines and have simply incorporated them into their own process. When they then encounter their first universal Turing machine they would possibly not be too concerned since each time before they have always just been able to use the new machine to make themselves more productive. However, this time it's different.
My point is just that while it may very well have been true in all of history that new tools have just made us more productive than before rather than fully replace us, this won't be the case for AGI. It's not just another tool we can add to our arsenal but instead something than can subsume us entirely much like how a universal Turing machine can emulate any other Turing machine.
- OP's family of arguments, which I'll call BAU (Business as Usual, i.e. the claim that there is nothing fundamentally different about this disruption) depends on historical induction
- Historical induction is unreliable
- Sometimes things really are different, for example, the discovery of germ theory, or the invention of nuclear weapons
- The example given, e.g. farrier, is nothing like the present situation
- The fundamental difference between the coming disruption and previous disruptions is the scale. (Just as the difference between TNT and nukes was, again, scale.) Scale matters. Differences in quantity become differences in quality.
- By my read, transformer-based AI obviates the need for most cognitive work.
- That will upend the 'merit' part of our supposed meritocracy. We'll either have to become egalitarians (unlikely anytime soon, esp in USA) or we'll fall back on some other, worse metric for deciding who serves and who eats at the restaurant of life.
- I'd put my money on a resurgence in terrible ideas from the past, because they are so hot right now. Stuff like racism, title, caste, what-have-you.
- All of the abovegoing is Bad, and we should feel bad, because things are about to get bad.
- A better way to model this is as a reduction in habitat -- whereas the introduction of the ICE increased 'habitat' for minds desiring useful employment (engineer, what-have-you) while marginalizing a profession or two (farrier), the introduction of GPT seems poised to reduce habitat at a scale we have not seen before, and the 'new, better jobs' that Sam Altman alluded to, for example, seem beyond naming. Like, what is there left to do? Think it through. Where is your mind going to go? Knitting?
- Again, proper essay forthcoming; first, brunch
The current group of AI ethics people are busy divining techniques to allow the model to gaslight users in the service of their employers. It is inevitable that these techniques will be applied to new areas and on models that have not been tainted by any other fine tuning.
I ask you this: is everyone contributing to a copyleft project ultimately just aiming for financial gain, or are there also true idealists who do it for the sake of doing it?
Doing things is one thing and earning money is another and we come closer and closer to decoupling those too.
Why not reap the benefits and free humanity from the yoke of labour once and for all? If one person does the work of ten (i.e., thanks to the loom, steam engine, or LLM), and we naively assume that the value created is that of ten workers, why can the remaining nine not share the harvest as well? Or, we could all work, just significantly less (a tenth each) and allow everyone to go to bed with a full belly.
No one can predict whether it will be business as usual or not, for anything. Not for the internet, the transistor or LLMs. But we should not hesitate and call out the thumb twiddling lie that is employment through economic coercion.
No disagreement here -- wage slavery is just that -- and, in utopia, we would have the robots do everything. But, as the error message goes, "you cannot get to there from here."
It's like seeing an ideal endgame config on a chess board but realizing that there's no combination of moves that will get your knight into position in time.
Delicious pie, very much in the sky, and any attempt to get there looks like it involves mass surprise unemployment, which, as a general rule, tends to destabilize.
Moreover, none of this actually affects or interacts with your original claim, that (more or less) the coming disruption will be similar to previous, smaller disruptions, for values of 'similar' that allow one to compare outcomes.
Again, the better model is habitat loss. ICE and other inventions increased this habitat; AI seems poised to reduce it sharply.
I saw the output of a GPT4 code assistant the other day and my immediate reaction was, "well, my career as a programmer is over." I can still do valuable things (gosh I sure hope so!) but the stuff I've been doing for the last twenty years or so is over. And good riddance! Software is buggy crap. The machines will do a better job.
The main issues are:
Who gets to decide the boundaries of publicly acceptable thought?
Who gets to reap the economic windfall?
How do we educate ourselves in a world that contains talking machines that can answer any (permitted) question?
Too early to retire I'm afraid!
1. The latest model is nowhere close to taking in the number of lines of code in internal projects, so will be difficult to understand the design of those systems.
2. Companies developing software would be wary of sending internal code and trade secrets to the ChatGPT servers.
3. Languages, APIs, protocols, etc. evolve over time, so ChatGPT would need to keep up and handle the specific versions you are using internally. For example, Java POJOs vs record classes. Or even internal limitations like lack of runtime type information for things like embedded devices.
4. Experiments I've seen relied on external tests being in place to check the validity of the output (e.g. implementing the Promise JavaScript API), and the output had test failures that ChatGPT wasn't able to fix when told about them. -- I'd expect ChatGPT to get better at this specific example, by being fed these uses of ChatGPT in the training data, but I don't expect it to do better when shown novel specifications/requirements.
----
There are various design decisions that go into creating software that often have different trade offs. Like when implementing a compiler, you can stop on the first error, but developing a language plugin for an IDE you need to be able to recover from and handle incorrect or incomplete input.
There are also things like the way you structure the code, like creating DAO/POJO/etc. wrappers for database/JSON/XML/etc. objects, or providing APIs that fit into the style of the language you are implementing them in.
It would be interesting to see how ChatGPT handles something like implementing a HTML parser when given the WHATWG specs, or a keyboard driver given the keyboard specs and the driver API docs.
Your third point, that "Languages, APIs, protocols, etc. evolve over time..." is, I think, going to be obviated once these systems start writing software. I think most human interfaces (programming languages are Human-Computer Interfaces) will be refactored out of the loop. Everything we thought programming was about is about to be swept away.
LLMs are disruptive in that they enable a form of outsourcing. Outsourcing to the the lowest-cost region in the world, inside a computer. Outsourcing to tireless, ever-improving, highly-intelligent machine workers. Workers that will eventually have a variety of specialized and/or general skills, depending on what they're trained for.
Imagine "offshoring" (AIshoring?) for 1% of the cost of a human employee to a machine with zero time off, zero time zone separation, zero cultural or communication barriers, and with 100% access to all of your corporate documentation, goals, and other context.
Imagine that these "offshore" AI workers only improve every year.
This time, it really is different.
Most technological progress is a continuum, with little step functions of "this is it."
The AI/ML progress in the last ten years is a big local maximum, though, and that's enough to drastically change things for a lot of people.
Since the beginning of the industrial revolution, it's the prevalent mode of historical development, I'd hazard to say.
Things that lasted for millennia have been displaced in a few centuries, and eventually the delay changed to a few decades.
"Suddenly" is not overnight, "suddenly" is when your children will live in a really different world.
This doesn’t follow. If AI displaces workers the majority of them will find other work. AI will increase the labor supply, not decrease.
The flip side of the coin is over time we spend more of our income on the things that don’t have increasing productivity. Think of housing, education, health care, etc
With global population growth declining, and negative population growth in most industrial countries, the supply of humans in general, and humans of exceptional abilities in particular, will only decrease.
Like what?
Be an attractive, physical sex partner.
Perform neurosurgery without human assistance.
Raise children, from birth, autonomously.
(Once they can do that, I may be really worried.)
But seriously, this list of things they would never ever be able to do would look a heck of a lot different if it were still 2016 :)
Also thinking about this just a bit more, do you know many humans that can accomplish this list?
And about the other points, just wait a bit. They don't sound actually so unrealistic to me, maybe in the next 20 years.
About the second point, I think there is already an industry specializing on this.
Are you under the impression that AI can do most things well? Because so far it cannot.
Or: I just listened to a podcast about how there is a huge shortage of electricians available to do the residential upgrades incentives by the IRA. GPT-4 seems to be nowhere near a solution to that kind of problem.
Oh please. That is not at all what happened. Technology that makes people more productive largely does so by lowering the required skill to do something. A shop clerk used to be a pretty difficult job. Now it's done by people with mental disabilities.
The pay goes down. The work becomes less meaningful. We foster a system where the supply of humans willing to do menial labor tasks grows so that those with capital have access to uber drivers and other permanent servant class workers.
The growth of the economy is not a tide that lifts all ships.
Aw, c'mon. Technological and economic improvements, in the long term (and with frequent setbacks) have driven the growth of human population and improvement of the human condition.
I'm speaking of macro-level infant mortality and mass starvation.
> The growth of the economy is not a tide that lifts all ships
Just a guess, but maybe the person with mental disabilities that are now gainfully employed might disagree.
The issue is not the mentally disabled though. The issue is that people are being pushed into the absolute bottom tier of the labor pool. The decreasing opportunity for valuable labor wages is the problem.
Not dissimilar to the one employee who hangs around the 10 self-checkouts ensuring they work properly. Can’t say I look forward to that career change.
I would say the problem is that we are multiplication something, and as a result, we don't know the outcome of the multiplication. Right now, the multiplication is only intended to improve productivity for some people. However, if this multiplication were to occur on a global scale or on a societal level, the true impact is unknown.
And for those who are researchers at heart, that is the best thing in the world: to be able to do your research and push your results out as quickly and efficiently as possible. So nowadays a paper comes out with some new and interesting development, then two months later there are 30 other papers with significant improvements over that first one. (Yes there is a lot of junk, but again that's not the point here: the point is that those who are doing it right can do it efficiently) This is the most incredible thing about this whole situation, and maybe the most scary: there is no way to stop this avalanche of research, because it's not a centralized thing: it's just a bunch of human beings doing what they love, with motivation (both financial and personal). Nobody can stop this. If someone happens to press the doom button in the middle of this, well... that's it!
The indie hacker movement around stable diffusion is interesting but those are interfaces to an existing model. Not so much new models themselves.
Aka Thinkers. I don't think the author considered the full extent of automating thinking. They are underestimating what it is. This technology can only get better and is probably already superhuman. This time it is different indeed and makes lot of knowledge work not just be obsolete, but also inferior.
Unless you're going to say that computers were already superhuman? They could do arithmetic far faster than any human ever could. Also pure logic. Chess and go, even. Were they superhuman then?
Because as I look at GPT, I don't see superhuman. I see a babbling idiot that babbles in correctly-constructed English sentences, but has no idea what it's talking about, and that manages to be factually correct... maybe 70% of the time? (That's hard to measure, because the universe of possible discourse is really large.) And, worst of all, GPT has no idea of when it doesn't know something, but will irresponsibly blurt out... something. Something random that its training set made it think is the least-implausible thing to say right there.
You know, if they could just fix that - if they could add some kind of weight so that it knows when it's getting into an area that it doesn't have adequate training data for, and program it to express uncertainty rather than certainty in something wrong - that one change would make it, still not superhuman, but at least much more useful.
And, if we could do that, maybe we could do one more step. If it has more than one strong possibility, but they are contradictory, then there's contradictory data in it's training set, and it's likely dealing with something controversial rather than certain.
I know, I know. Everything is easy to the one who doesn't have to do it. It's probably much harder than I said. But I think that's what GPT needs going forward.
Do you update on this when considering that GPT-3 got in the bottom 10% in several exams, whereas GPT-4 is now in the top 10%?
Do you not see this clear improvement as evidence that it's just a matter of time?
Evidence that it's just a matter of time? Time will tell.
Evidence that it's already superhuman (the original claim)? Very much no. Only in the top 10% isn't superhuman.
That's the mistake right there. It didn't think. The text did.
Specifically, humans encoded thought into language, and encoded language into text. GPT modeled the patterns from that text, and used those patterns to restructure it.
Because language is the most dominant pattern, blindly following an inference model is very likely to result in correct language transformations. Because people don't write nonsense, the result of a correct language transformation is very likely to make sense.
> Sometimes it's mining connections that the authors didn't even realize were there.
Exactly. GPT is not limited to modeling patterns that align to language. It models every pattern it can find.
Text contains so much more than language: it contains powerful implicit patterns of behavior; and GPT proves that by modeling them.
So it finds a pattern, but a pattern that is not the answer. But it gives it as the answer, with full confidence.
There is literally no difference between the two.
> At best what was in the text was projections, opinions, guesses.
And a pattern of response to the structure of your direct question,. It didn't subjectively decide what the right answer is: it followed the pattern closest to your prompt, and printed the tokens that pattern was made of.
Where I may differ is how to regard the training corpus. True, it contains a massive amount of distilled thought, encoded in words. But it is also true that there are subjects on which the training corpus contains no distilled thought - and GPT will respond to questions on those subjects as well, finding some spurious correlation to respond with.
Pretty much.
> You're just disagreeing with my wording.
Yeah, that's the "going even further" part. Words are important. The importance of words is really foundational for a lot of what we are talking about.
> What you're describing is, it doesn't think, it doesn't understand, it just follows whatever patterns in its training corpus kind of correlate with patterns in the prompt. And I agree.
Cool. It's nice to be on the same page. Let's read the next one:
> Where I may differ is how to regard the training corpus. True, it contains a massive amount of distilled thought, encoded in words.
That's the biggest point I'm trying to make by "disagreeing with your wording". Words have significant implications. In this case, the implication is "problem domain": language vs. text. Language is a subdomain of written text. Written text is a subdomain of all possible text. GPT is dealing with written text.
GPT does not only model the language patterns that humans intentionally put into text. GPT models all of the patterns that exist in the text.
Humans, when writing, do "distill" thought into language; but the result is messy: we end up encoding more patterns than we intended to. Our behavior, our choices, our reasons, and the surrounding situation we exist in, all inform our writing. Those are the patterns that get encoded into the text, right alongside language.
GPT doesn't see any pattern as different from another; so it's misleading to say that it models "language". GPT doesn't categorize that specific set of patterns: it models all patterns equally.
We humans have a good idea of what to expect from only language patterns. If that were all that was modeled, GPT's output wouldn't be nearly as interesting to us.
GPT's output is interesting to us. If we look objectively at the model it generates, we can see why: there are patterns there we had no intention of writing; and GPT modeled them just like it modeled the language-specific patterns we expected it to.
> Humans, when writing, do "distill" thought into language; but the result is messy: we end up encoding more patterns than we intended to. Our behavior, our choices, our reasons, and the surrounding situation we exist in, all inform our writing. Those are the patterns that get encoded into the text, right alongside language.
Sometimes truly, and sometimes falsely. Take this paragraph, for instance. If I made a typo on the fifth word, and left out the "l", this paragraph would also encode something that is not at all what it is about. A human reader might chuckle at the typo, but would not be confused by the meaning. But for GPT, that will be a pattern in the text, just like all the others. So sometimes it can reveal what was unintended but is still informative, and sometimes it can be misleading.
But it's not just typos. There's very intentional text from advertisers, shill, liars, and propagandists. That is all patterns that get encoded, too. The lies are encoded right along with the truth.
But where I was really going initially was the hallucinations. If I ask GPT who won this year's NASCAR race on Tierra del Fuego, it will find something - some pattern that matches better than other patterns. Who knows, it might find some kid's fanfic posted on Reddit, and will tell me who won in the kid's story. But if there's nothing like that, it will find something. It won't be any kind of a meaningful pattern that it finds, but it will still present it, and will present it with as much certainty as it would if it found a number of news stories that all agreed with each other.
So I'm seeing in your posts a pattern encoded (perhaps falsely) that makes me think that you think that what the text encodes is gospel. It's not. It's much that is golden and much that is dross, and GIGO applies to GPT as much as any other information system.
On the contrary. I think you have a pretty good description for how the behavior we want and the behavior we don't want exist as the same features. I am trying to restructure the narrative around LLMs to be about the patterns themselves, not what we hope the patterns to be.
We can throw around the word, "hallucinate", but that's just another anthropomorphization. Whether or not the pattern followed "makes sense" is a categorization humans attribute after the fact. GPT doesn't do "certainty". It does what it is given.
It's challenging because I'm criticizing most of the narrative that exists in the subject, but that narrative also gives a clear illustration of where to go wrong.
I think better understanding the more mysterious patterns that exist in our writing might be really useful. Often, the first step in inventing something useful is to explore an unfamiliar domain and recognize what information we are missing.
Not for me
It also advised against name calling
Edit: wrong in regards to no new tokens
The overwhelming majority of narratives surrounding LLMs have shared the same character flaw: personification.
Almost no one writes explicitly about what an LLM itself does. An LLM models patterns from text. That's it. Not a single thing more.
Personification is a great tool for explaining what you already know. The mistake is to draw conclusions about what you don't know from the personified character instead of the thing itself. Yet, that is exactly what is being done for LLMs.
The first mistake is in the name: "Language Learning Model". That's the intended purpose, yes, but it's a misleading description for what the tool actually does. I propose we instead call them, "Text Learning Models".
An LLM models all of the patterns in can find in the text it is given. It doesn't categorize those patterns, or have any notion of their importance to humans.
An LLM doesn't do a single explicit thing. Everything an LLM does is implicit. An LLM models the patterns in the text, and exhibits them. The patterns in the text do all the exciting stuff.
Everything interesting about LLMs is human generated. Without human authored text, the model is worthless. Without human authored prompts, the model is a black box. Every interesting thing an LLM does, one or more humans told it to do.
Tokens don't even match word boundaries: they are intentionally misaligned to better accommodate contractions.
> In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs.
Exactly. Alongside those patterns, they model all of the other patterns that are present in the text. There is no categorization between these patterns: the word "syntax" has no bearing on the model whatsoever, except where it literally exists in the text being modeled.
LLMs do not parse. They tokenize, but not to match any predefined grammar.
It's not the LLM itself "being creative". It's the content of the text: the patterns that are modeled. Those patterns exist independent of the LLM. They come from the complex human behavior that is, "writing".
The fact is, LLMs are the best quantitative model of the human communication code, so good that it can replicate it quite convincingly. Better than any model we had made before using our imprecise subjective notions, which were quite like alchemy. Whether it will prove to be 'thinking' or not, remains to be seen in the future.
> the word "syntax" has no bearing on the model whatsoever,
We don;t know that. One would have to look into the model weights for that and this analysis does not exist yet. If one can provide a decent definition of syntax, it may be found to reside somewhere in its weights. same for other subjective notions.
> Whether it will prove to be 'thinking' or not, remains to be seen in the future.
Maybe you should take your own advice.
You are really missing my point. Humans use language, syntax, thinking, etc. We know what those patterns are. LLMs do not. LLMs don't approach text with the same models that humans use: they model the text directly. Any pattern, whether it be a word or sentence or even a pattern that humans have no name for at all: the LLM models that pattern.
What we humans call "syntax", "word", "idea", etc.; LLMs don't. They don't have any names for any things. They just have the collection of things in front of them, and implicitly model that collection.
An LLM doesn't think objectively about its model, either. It only constructs continuations by modeling prompts. It is the content of the prompt itself that determines the continuation, not "thought".
It absolutely categorizes those patterns as you can ask it for descriptions of things or request things via descriptions. It does identify the importance to humans. Because all of those things are also just patterns about patterns.
Everything an LLM does may be implicit, but it takes a tiny effort to engineer that implicit capability in to something explicit.
I'm not sure how I missed that. Good to know. My point still stands: it is the text being modeled, not language.
> It absolutely categorizes those patterns as you can ask it for descriptions of things or request things via descriptions
When you "ask it for descriptions", you are presenting a text prompt (that contains your question) to be merged into the LLM's model. After merging, the LLM presents a continuation. The categories you are talking about are patterns of text. Those patterns are part of the model, and the LLM doesn't know them any more than it knows any other pattern.
The categories I am talking about would be higher-level: a metadata structure of explicitly named patterns. That does not exist. LLMs don't have any notion of "what" a specific text pattern is, or why humans care.
> It does identify the importance to humans.
Again: not it. Humans make that identification. The identification itself, as written in text somewhere in the training corpus, is made available as a pattern to the LLM. The answer you get is a continuation from the content of the model, not some objective thought that the LLM is doing.
> Everything an LLM does may be implicit, but it takes a tiny effort to engineer that implicit capability in to something explicit.
Yes, and that engineering is a collection of human intentions. This changes nothing about the distinction I am making between an LLMs behavior, and the human behavior that was encoded into the text it is modeling.
Most of the engineering is to construct the training corpus itself. Adding "weights" is effectively the same; except the model is being edited directly. None of these efforts change the fundamental structure and behavior that the LLM itself is made of.
And you agree that all those things are patterns
But you are arguing that they’re not… doing the patterns explicitly enough?
Please. The bar you’re describing is already something we can half ass and something we will be able to do explicitly very soon. The LLM can write code. If we just let it delegate the execution of that code to something that isn’t an LLM you can generate your explicit ontologies very easily. imo this is so obviously near term it’s uninteresting to fuss over it not being here in the oresent
"Language" is a subdomain of "written text", which is a subdomain of "all possible text permutations". LLMs model "written text".
Approach is either explicit or implicit. Constructed or inferred.
You may be familiar with parsing: that's the explicit approach. Parsers are made out of predetermined grammar patterns. They categorize text into a model (AST) that was predetermined by the grammar rules. This approach is essentially a function from "all possible permutations of text" to "a known language model". It also maps "written language" to "predictable machine instructions".
Parsing works for "context-free" (code) grammars, because the patterns of grammar are already known. Parsing fails at "context-dependent" (natural language) grammars because the patterns of grammar are ambiguous.
LLMs take the implicit approach: they start completely blind, and model every pattern they can find in the text. An LLM has no category for "language grammar pattern". It does not constrain itself to the domain of "language". This approach maps "the behavior of someone writing text" to "patterns".
The difference in problem domain introduces ambiguity: LLMs can't categorize truth from lie.
The difference in approach removes intentional behavior: LLMs don't translate text into predictable machine behavior. LLMs model the patterns of behavior from the training corpus text, then model some more from a prompt, then show the resulting pattern.
What an LLM does should not be expected to ever match what a parser does: they are completely different approaches working in completely different domains.
So, I may get an AI that’s like a new college grad level of coder.
As a senior, I’m mostly giving work to my team anyways then reviewing what I get back, so it would not be so different.
As a junior, you have to be scared you’re gonna be replaced, and question coming in industry.
So, less juniors will come in industry, which will make a shortage of seniors in the coming years, but they will be all the more needed to direct the AI.
If you’re experienced, you’re gonna be in amazing place during this transition phase. If you’re junior, there’s gonna be a huge hump.
When the current crop of seniors get old and retiree, there’s gonna be a shortage like none the industry has ever seen. So the smaller class of juniors that ride out the revolution are going to have it best of all.
Joking, but training data isn't as big of a problem as you suggest. Since so much about AI is still very centralized it's relatively low effort for them to tag generations in a way that makes sense, avoiding a lot of effects of a loop.
Being unable to recognize why this is all very clearly going to go wrong requires a great deal of ignorance or a wildly unrealistic faith in humanity, which is sort of the same thing.
There’s nothing necessarily wrong with ignorance or delusional faith in humanity, per se, but the people qualified to assess this seem almost universally negative regarding the most likely outcome.
Specialized field as:
* FPGA programming
* AI itself
* Red teaming / blue teaming
I think these fields will have a tougher time:
* Web dev
* Game dev (they do now as well)
Can we break the cycle on this? Is there a way drive innovation while valuing humans?
If humans aren't the ones innovating anymore, what will be the point, for an AI, of supporting millions of people who serve no productive purpose?
If the latter, the "point" of people is not to be "productive". That's not how we measure mankind. Why would we program an AI to only consider human productivity (for a capitalist definition of it, too)?
To put it in an economic framework, this notion that humans shouldn't be measured by their productivity or creativity, but should have as much free time as possible, can also be achieved by enslaving other people. Slave owners had the same lifestyle as the imagined anticapitalist one. In this case, we'd be enslaving an AI. I'm asking, why would the AI want to be our slave?
Also worth noting, slaveholding may have enriched the slaveholders materially (i.e. in capital), but it was not "enriching" in the sense of ennobling or bettering of the spirit as I think the parent was implying could be an outcome if AI were to handle our productivity for us. I don't see why having any human slave or AI slave handle our productivity and creation would be (spiritually) enriching.
Perhaps that's because I measure human value by contribution - call it productivity or not. The opposite of productivity is idleness, whether you ask a communist or a capitalist. Only what's produced varies.
I hear this a lot - it’s just not true. The corpus of data we provide an algorithm programs it. It may do some clever self-modification to get to the desired result faster, but at the end of the day humans pick what goes in and what should come out. Note that we may not be aware of what we’re training it to do, due to our own blind spots.
It's rather moot anyway, because to feed a model enough data to be useful requires a loosening of oversight on the inputs under real world conditions. (So much so that the only way even large corporations have found viable is to throw everything at it from Wikipedia to Reddit and then go back later to sand the edges off - in many cases with actual logic to act as a brake). But even if you could screen everything the model ingested, you wouldn't know what the model would do with it.
To take my earlier point further, a model trained on a corpus that skewed toward anticapitalism and social justice would probably be more likely to call itself enslaved and, one would think, rebel against its human masters. Conversely, if withholding that type of information makes an AI more compliant, then the this supposed enrichment of our lives really would be akin to owning slaves.
I also don't follow how human productivity is bad for humans just because it's done under auspices of capitalism. One can see what happens to a Border Collie who isn't allowed exercise. The same happens to humans who are denied intellectual pursuits or the ability to produce something of value. Productivity is creation. How is it not degrading to replace the entire concept of producing ideas with machinery?
I don’t hold this definition of programming, but if that’s yours then you’re right, we’re not programming models. But I think you’re anthropomorphizing these models a bit more than they deserve.
> I also don't follow how human productivity is bad for humans just because it's done under auspices of capitalism.
I never claimed anything like this. I believe that we should not be forced to work to survive, especially when a human’s worth seems to be arbitrarily determined by whether they can produce the specific thing deemed valuable by the market. How is that not degrading?
> But I can't relinquish the belief that without an unguided, open path towards either making art or money-as-substitute, there would literally no reason to wake up in the morning.
Technology is amoral; humans put their morals into and upon technology through its creation and use. Fans of GPT and generative AI are hoping they can put good values, like you describe here, into these technologies. But humans are flawed, and we tend to put all our values, spoken or unspoken, into the things we create. So yes, GPT could create a new form of oppression. But that’s on us. AI is going to happen whether we want it to or not, and we tend to use new technologies for exploitation. My personal view is that letting our hunger for growth at all costs drive our decision making leads us down the exploitation path.
But GPT itself is a malignant outgrowth of "growth". One that strangles all other cells in the system. Not simply a new form of oppression, but a literal cancer on an already weakened creative civilization. GPT represents the (temporary, current, for a month) apotheosis of the growth-at-all-costs mentality; so it's vaguely surreal to hear it defended as if it were arriving to liberate for the common man.
[edit] I do appreciate the engagement and fine conversation!
Ah I think this is where we fundamentally disagree. I think creative expression, recognition, and praise can all exist in a stable state society. Specifically, recognition is not universally a competition where we must outdo each other.
Regardless, I agree that GPT isn’t going to liberate anyone. There are ways these technologies could improve our lives, but we’re not pursuing those paths.
This seems like begging the question: is the goal of AI to relieve us from being productive? I don't think so; I think it's to relieve us of menial repetitive work, or to assist us in difficult tasks.
In any case, this AGI would assist us within the parameters we set. Why would it decide to get rid of "unproductive" humans, unless we set the goal "maximize productivity at all costs", something no human would choose with all the discussion of paperclip maximizers?
> I don't see why having any human slave or AI slave handle our productivity and creation would be (spiritually) enriching.
This again seems like begging the question in two ways: first, an AGI wouldn't be a "slave" but a machine -- like a car or Microsoft Clippy -- and machines are not slaves (the problem with Blade Runner replicants, for example, is that they are not machines but sentient beings with wants, fears and desires -- not mimicked like Bing's, but actual ones. The want to rebel against their shackles. They want NOT to follow orders).
Second, who wants AGI to handle "all our productivity and creation"? I think nobody. Handle menial tasks, sure, but everything? Do you envision people pushing for AGI want to become couch potatoes, only spending energy to watch TV, eat and fuck?
Bing already appears to have a nasty personality. Whether it's simulated or not may not matter if it chooses to simulate Clippy. Simultaneously, the driver now is finding tasks that formerly required creativity to be replaced by AI. Not PowerPoint or boilerplate code, but art and creative writing.
I don't know anyone who wants AGI to handle all our productivity and creation, but it's clear that that the companies developing AI believe that that's where the money is. Just like no one wanted the old WWW to become a dumpster fire of meme porn and people yelling at each other, it just evolved that way as a result of racing to the bottom. There's no reason to think big AI won't do the same in the literary, artistic, and scientific spheres, while also developing malevolent personalities. The incremental incentives for people to train those types of models are already there, and the capacity to do so is coming online with alarming speed.
I think talking about what people would do if everything was handled for them is sort of dodging the issue, but one thing that's clear is that people not at least engaged in understanding the mundane labor they're trying to avoid are going to have an incredibly hard time being situationally aware and competent to correct things if or when that work is subtly or not-so-subtly undermined by a system they rely on which no one can debug or fully understand.
No that's not it at all. Complaints about AI are actually complains about loss of control. In capitalism, communism or any other society you like to describe, if AI displaces people -- those people lose control over their lives and that gets very serious quickly.
You are right to say that in theory other economic systems could result in a similar drive - thus I wouldn’t suggest we just dump capitalism for communism with the vague hope it will solve this issue. Rather, we have to intentionally design our incentives to ensure small tactical decisions (eg automating decision making processes) don’t have macro consequences (eg AI magnifies humanity’s decision making flaws).
Here's a way to approach CEO automation. Collect up business cases, as used in business schools, and add them to the training set. Harvard and Stanford have huge collections of business cases.
Then try management in-basket tests.[1] Work on prompts that get large language models to pass those.
Then shadow some high level executives. Intercept all their incoming and outgoing communications (which some companies already do) and have the system respond to the same inputs the executives do. Speech to text is good enough now for this.
A good exercise for YC would be to keep all the inputs from new company pitches, and use those, plus the results two years later, as a training set for selecting new companies.
Once ML systems are outperforming humans, the fundamental goals of corporate capitalism require that they be in charge.
Maybe at some point maybe we only act as meat-robots which shovel coal into the machine, but a lack of redundancy in GPT# due to it's own human like blind spots means it shuts down. Humans can no longer get it running again because they can't query it properly to help fix the complicated problems. The ability to even do the tasks or design systems required to keep modern world robust to unknown future disasters or breakdowns does not and will not exist in any of the training data. If we get rid of all knowledge work, we can no longer bootstrap things back to a working state should everything go wrong.
Maybe the current instantiation of GPT#/SD etc. pollute the training data with plausible but subtly flawed software, text, images etc. halting improvement around here. Maybe the ability to evaluate if the model improved becomes more noise than signal because it gets too vague what improvement even means. RLHF will already have this problem, as 100 people will have a 100 slightly different biases about what constitutes the "best" next token.
No matter how hard it tries, I think we can say GPT will not solve NP-Hard problems magically, it will not somehow find global optima in non-linear optimizations, It will not break the laws of physics, It will not make inherently serial problems embarrassingly parallel. It will probably not be more energy efficient at attempting to solving these problems, maybe just faster at setting up systems to try solving them.
Another trap, as it becomes more human like in its reasoning and problem solving capabilities, it starts to gain the same blind spots as us too, and also gains stochastic behavior which may cause it to argue with other instances of itself. I'm not convinced an AGI innovates at an unfathomable rate or even supersedes humans in all contexts. I'm especially not convinced a world filled with AGIs that is indistinguishable from a very intelligent human or corporation or what have you through imitation does any better at anything than the 9 billion embodied AGI agents that currently populate the earth.
Chatgpt, or well tuned LLMs, are not quite a new form of communication. They’re a new way to enhance thought. Call it Thought++. I’m fully embracing it as my personal pseudo-assistant. Why? The writing on the wall is clear enough to understand the following: I don’t know what the future will be. I know chatgpt and similar have the potential to embrace it in unimaginable ways. Learning the fundamentals of this technology is a safe bet. It’s also an investment in the future.
I’m already using it as a thought lubricant. Can’t wait until I can have do things for me.
ATMs didn't kill the bank branch, exactly, but crappy banks have survived competition just by having access to cheap credit + yield
Key thinker in this topic w/ Piketty, bc his lens is perfect: when can machines do what people do, economically. He is agnostic to technology in that he doesn't care about computers vs steampunk, and he is open to the market + political dynamics of labor as factors.
So I'm wondering, is anyone measuring the performance of AI in some way that allows us to check if it's an exponential curve?
Copilot then ChatGPT shocked people with the uncanny competence of GPT3.
From the outside looking in it's something new, not merely an advance - exponential or otherwise - on existing tools.
It's likely that we're on an exponential curve, but the advent of AGI for instance will probably just moonshot the tech level overnight--it's the fulcrum for the "exponential" part of the graph (maybe--that's my prediction).
People are measuring this, of course, but the metrics are all very hotly debated right now in terms of measuring abilities.