Unpredictable abilities emerging from large AI models
quantamagazine.org
quantamagazine.org
I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models.
I was walking this morning with a non-tech friend and I was trying to impart to him my feeling that all knowledge work, and a lot of mundane work is getting disrupted. Even though I have many tech-friends who are "Large Language Model doubters", my friend got it this morning. Emotionally and psychologically I think that some tech and other highly paid knowledge workers just can't accept the sea-change that we are living through.
For myself, I am spending a very large amount of time experimenting with the OpenAI APIs, LangChain, and Llama-Index - and I am enjoying myself tremendously.
Why overproduce?
The average person might not care for this, but the psychopaths who crave (and are in) power do...
This is a good way to frame it. As a mid-career person, I’m trying to figure out how to respond to these developments in my own career.
I think there’s a good chance that software development as a career looks very different in 10 years in terms of the day to day work and opportunities for employment.
25 years ago in university, I did roughly the same stuff as I do today. Languages are higher on the abstraction ladder, networks and hardware are faster, etc, but stuff is still roughly the same. I can see a real discontinuity in the work on the horizon.
You have to learn your IDE, but it doesn't make programming harder. It makes it more fun actually.
Chatgpt is going to require that devs are even better IMO because you have to catch its errors and know how to work with it.
But if you work at BigCo and mostly sit on your hands already, then, yeah. I don't know.
Does it? Slowing our breakneck anti-human business pace and stopping mindless and environmentally and culturally harmful consumption and production of trinkets is surely better!
You mean, spending >1/2 their budget on police?
Well, let’s hope not.
Will that result in less demand for software engineers? I doubt it. I think it will lead to companies and organizations doing way more software engineering.
I was going to use the old "ATMs resulted in more jobs for bank tellers example", but apparently that particular example is flawed: https://www.vox.com/2017/5/8/15584268/eric-schmidt-alphabet-...
Um, yes. Been in a bank branch lately? Had to visit a bank branch for any reason?
Every innovation can be used both for good and for evil, and it often has unintended side-effects. Just ask Thomas Midgley Jr., who played a major role in both leaded gasoline and CFCs. Or Alfred Nobel, who tried to make mining safer, only to be appalled at an erroneous obituary to describe him as the "merchant of death" due to its military use.
Over the last two decades we've seen tech giants go from a bunch of nerds in garages to parasites invading our personal lives and consuming every bit of data about us they can get their hands on - and social media went from a cute gimmick to a society-destroying tool powering genocides.
AI is undeniably extremely promising, and it is not hard to see that it has the potential to change the world forever. However, we are inherently unable to understand how it works, or to alter it in a meaningful way. It's a Pandora's Black Box: will it give us a better and easier life, or will it destroy us? Nobody knows, least of all the people who are opening it!
Soon to be 60 year old here. Glad that if need to I can retire now. Certainly the rise of LLMs and generative AI isn't going to be all bad, but I've also got a feeling that not as many software developers will be needed soon since those who can leverage LLMs will experience a pretty decent productivity boost. Part of me wonders if at least some of the layoffs we've seen in the last couple of months are because companies such as Google, Microsoft, Amazon, etc. (the ones that have pretty clear visibility into what's happening in the AI space) are realizing that they aren't going to need as many knowledge workers in the not-so-distant future.
I think there was always this idea in the back of our minds that this was going to happen someday. But someday was always like 15 to 20 years out. Looks like someday is knocking on our door now.
OTOH, software development will become significantly cheaper. That means the business case for using software in more places just tilted in favor of throwing some software at it. You'll see more and more businesses using it, ones you didn't expect, like local mom and pop businesses.
Yes, it's going to be different but I don't think we know exactly what is going to happen yet.
And in a lot of cases they're not going to need to hire a software developer to do develop it.
One of the reasons the work at Big Tech is so fun is because they actually need to engineer a lot of things. Smaller tech companies or businesses that utilize tech can just glue a few libraries together to make it work.
The average human is only slightly smarter than the average rabbit. They are going to pay someone who specializes in this task.
In the decade(s) since I kinda think "Chomsky was wrong" is pretty much the only reason people bring him up.
When Chomsky proposed generative grammars, his theory of universal language acquisition, and so on, they were radical ideas that upturned the central cannon of linguistics.
Time has been on his side - entire schools and subfields of linguistics went extinct as more evidence emerged that Chomsky was fundamentally right. Basically every computer language and data format in existence is parsed/lexed in ways inspired by his models of language.
But now Chomsky is considered the stodgy old establishment, and whenever one of his theories is contradicted somewhere on the margins people shout "Aha! He was wrong the whole time!" and ignore the 99% of cases where his models are still the best ones we have.
https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chat...
But might be missing something more specific, would love to read that.
Here's an argument otherwise, that they really are developing some sort of model: https://the-decoder.com/stochastic-parrot-or-world-model-how...
“ Their deepest flaw is the absence of the most critical capacity of any intelligence: to say not only what is the case, what was the case and what will be the case — that’s description and prediction — but also what is not the case and what could and could not be the case. Those are the ingredients of explanation, the mark of true intelligence.”
Isn’t that just plainly true of LLMs? Sure they can produce text that looks like an explanation, and sure putting in a header telling it to spell out its reasoning will get it to output text that looks like reasoning, but I feel the nature of its hallucinations make it clear that it’s not actually performing those steps anywhere in the net.
But maybe I’m just a Luddite? These technologies are amazing and transformative, I’m just shocked to see hate on HN of CHOMSKY of all people, the father of modern linguistics and cognitive science…
The communism support doesn't seem to quite stick either if WP is to be believed: https://en.wikipedia.org/wiki/Political_positions_of_Noam_Ch...
In Britannica and WP he's labeled an anarcho-syndicalist, so seems to be against both capitalism and noncapitalist authoritarian systems.
https://en.wikipedia.org/wiki/Cambodian_genocide_denial#Chom...
Did you skip my first link or are you disagreeing with it? It's about the same subject. And in your WP link has a second quote writing about the genocide as fact.
Another I think credible seeming article about it that seems to judge against denialism in an academic journal called "Genocide Studies and Prevention": https://digitalcommons.usf.edu/cgi/viewcontent.cgi?article=1...
Chomsky's skepticism in the media criticism article talking (among other things) about a book about the then-contemporary events of course looks quite unfortunate in retrospect, but given the volume of work and constant writing about world events over a long career it seems to be a error in judgement in the moment, rather than denying history, especially as he evidently updated his views later as more evidence came about.
It was completely unjustified at the time. There was absolutely no reason to doubt the testimony of refugees coming from Cambodia, other than they didn't fit into Chomsky's strange worldview that the US is the cause of all problems in the world.
Even today his views on Russia's invasion of Ukraine are completely moronic.
https://foreignpolicy.com/2022/12/22/russia-ukraine-war-left...
For ‘Peace Activists,’ War Is About America, Never Russia Their own hard-left worldview is so absorbing that they will take the side of any aggressor in the anti-Western camp.
https://www.e-flux.com/notes/470005/open-letter-to-noam-chom...
[0] Until we reach AGI, but I don't dare attempt to predict when this is going to happen.
One question out of academic curiosity: I'm exploring ways to use these tools for research projects in econ and am struggling to see a good angle.
For instance, suppose I have lots of PDF reports on how firms have evolved on each quarter (10,000 reports or any other number beyond what I can read).
Can LLMs be used to spit out variables based on these reports? EG: indicator variables (optimistic-vs-pesimistic), categories, etc. that I can then use as data to test economic models?
I tried simple cases with ChatGPT3.5 and got the feeling it's great for outputting narrative text, but felt mediocre for when the output was more narrowly defined into categories (which was surprising).
> One DeepMind engineer even reported being able to convince ChatGPT that it was a Linux terminal and getting it to run some simple mathematical code to compute the first 10 prime numbers. Remarkably, it could finish the task faster than the same code running on a real Linux machine.
Following the link, there's a screenshot to a screenshot [0] of a code-golf solution to finding primes which is quite inefficient, and the author notes
> I want to note here that this codegolf python implementation to find prime numbers is very inefficient. It takes 30 seconds to evaluate the command on my machine, but it only takes about 10 seconds to run the same command on ChatGPT. So, for some applications, this virtual machine is already faster than my laptop.
So it's not quite calculating primes; more likely it recognizes the code as being code to do so, and recites the numbers from memory. That's interesting in its own right, but we won't be running Python on an LLM for a performance boost any time soon. In my experience this interpreting is apparent as a limitation of the model when it keeps insisting on broken code being correct, or having its mistake pointed out, then apologizing, saying it's got some new code that fixes the issue, and proceeding to output the exact same code.
[0] https://www.engraved.blog/content/images/2022/12/image-13.pn...
We're building systems capable of programing computers non-deterministically. I think this is huge. But not because ChatGPT23 will be a CEO or a politician. But because this is a paradigm shift in compute similar to moving from integrator machines to general computers. I don't think LLMs will make programmers obsolete. I think large enough models will make programming something completely different from what it is now.
The days of sequencing tokens for compilers/interpreters seem to be drawing to an end as the dominant way of specifying software products.
But when it comes down to it everything in life is just nth dimensional tensor puzzles. What I really cherish computers for giving me is fast and clear feedback.
Instead, it is constant and accelerating destabilization. Hundreds of AI projects attempting to ride this accelerating wave were essentially just made obsolete yesterday - https://www.youtube.com/watch?v=DH-2BHDYNfk
I feel the excitement is going to very soon turn into frustration of attempting to remain relevant ahead of the accelerating technological curve. Humans need periods of stabilization to plan and reason about their lives.
That prime number example is a little bit like when you put a functools.lru_cache decorator on a function in Python. It's faster than computing the function call because it's able to recall the return value for the parameters from the cache "memory".
Of course, many skilled programmers are also mainly used a cache for common solutions to common problems organizations have in the programming domain. As humans we can derive satisfaction from being able to tackle the same task others can, as a confirmation of our own progress. We like "doing it ourselves". But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit. I guess that's the push-and-pull (and the anxiety generator).
In my opinion Stack Overflow does a fine job at that. And it's transparent in terms of solutions proposed being voted on and discussed. Turning that into sausage advice is a downgrade.
>But globally that's not super interesting if lots of people are constantly re-developing the same solutions to the same problems for their own benefit.
I'd argue this is how we train individuals thus globally quite relevant.
[1] https://en.wikipedia.org/wiki/Stochastic_dynamic_programming
The LLM didn't run the code, it tried to predict the output based on its knowledge of python and primes.
For shits and giggles, I just just wrote a series of `if...then` statements for 0 to 5, and only actually calculated the factorial for inputs >=6. I passed after the examiner just input 3 and 5 and was satisfied that the output was correct. Oops.
What would it do if the training data was only code? Only code+inputs+outputs? Only function signature+inputs/outputs? Only signature+comments?
Etc.
That doesn't change the fact that its predicting the output based on its knowledge of python though.
So, yes you're right, but at the same time it doesn't change anything IMO.
Correctly produces a convincing output for having decoded the base64-encoding of "ping google.com" and then piping it to bash: https://media.infosec.exchange/infosecmedia/media_attachment...
Similar command, but with a garbage domain it hasn't seen before, and a less well-known domain. It produced convincing output in both cases: https://media.infosec.exchange/infosecmedia/media_attachment...
Having it just output a base64 decoding of an obviously unique string. Fascinatingly, it tried to correct typos that I intentionally included: https://media.infosec.exchange/infosecmedia/media_attachment...
This was also pretty cool -- ask it to start a webserver then curl it: https://media.infosec.exchange/infosecmedia/media_attachment...
Telling it that it is a python interpreter and calling self-defined functions: https://media.infosec.exchange/infosecmedia/media_attachment...
A slightly more complicated function: https://media.infosec.exchange/infosecmedia/media_attachment...
I did a few more experiments including generating large factorial numbers that took a long time on my laptop but it responded accurately to a much larger length than my laptop could do (though these were only accurate to the first few hundred digits)
Edit: the base64 decoding examples are terrifying. I have no idea how that works.
Edit2: actually I can sort of see how that works, b64 encoding doesn’t have any obfuscation to it, I could see how an ML model can build a pretty good approximation over time after seeing all the examples on the internet.
Since we don’t know how it works, we should be open to the possibility that it’s using all sorts of storytelling techniques.
By the same reasoning, I hope people do realise that there are computations that are just impossible given the finite depth in a Transformer. The only possible way to overcome this in the current paradigm is the Chain-of-Thought related methods.
That it bears so close a resemblance to actual thinking says more about the importance of language to cognition than the other way around.
> The specific engineering of ChatGPT has made it quite compelling. But ultimately (at least until it can use outside tools) ChatGPT is “merely” pulling out some “coherent thread of text” from the “statistics of conventional wisdom” that it’s accumulated. But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of thinking behind it) are somehow simpler and more “law like” in their structure than we thought.
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
It can solve a physics problem in Telugu close to as well as in English.
This is key. ChatGPT/GPT-4 alone are limited to reformulating what they know from their training data. Linked to search engines, databases, and computational tools such as Wolfram Alpha, they acquire much more capability. We're already seeing that with Microsoft Bing.
(Update: what happens as large language models learn Excel? Especially since Microsoft is already connecting them to Excel.)
What's striking is how fast this field is advancing. Huge advances over months, not years or decades.
We now have a much better idea of how intelligence evolved. It's mostly just more neurons. One of the great philosophical questions has, inadvertently, been answered.
Is the Singularity happening right now?
That's not true. They are extrapolating. If they weren't, they wouldn't have the problem known as "hallucination".
No, but it seems everyone loves to LARP it anyway.
I see this as analogous to the human brain; there are different structures which are particularly good at specific tasks/functions. They all work together.
The only difference between a human brain and an ANN is a difference of degree. A neuron and an artificial neuron are functionally identical. I think as we start interconnecting these models we see surprising emergent properties.
This does not mean that we humans might predict all the time, in fact I would argue that LLMs only predict during training. They generate otherwise. We might also learn by trying to predict. I can imagine babies doing it.
I don't understand this use of "statistical" as a diminutive to describe these models.
Why can't incredibly complicated behavior be emergent from matrix multiplication subject to optimization in the same way that our biological matter has developed complicated emergent properties also being subject to optimization?
The loss function is very different, the optimization techniques as well, but the fundamental idea of complex behavior emerging out of a substrate subject to optimization seems common. I haven't seen a single good answer to that
LLMs are trained to reproduce human text, that is different from for example AlphaGo that is trained to win Go games. Trained to reproduce data is what we mean with a statistical model, trained to win is how we got superhuman performance before, while trained to reproduce data performs worse than the original creators of the data.
You're the one supposing a thing, so the burden of proof is on you. You need to demonstrate that the "incredibly complicated behaviour" that you're referring to (I assume this is longhand for "thinking", but please correct me if I'm wrong) is indeed emerging from matrix multiplication. Especially given that what you're suggesting is unexpected, given the known way these models work and the explanations that have been put forth already that extrapolate from the known way these models work.
If science were so credulous as to accept the first proffered theory about a new development, well, we wouldn't have these interesting AI models in the first place!
What do matrix multiplication and optimisation have to do with the way the human mind, or the human brain work? That they have anything to do at all, is your assumption, that you seem absolutely convinced about- and then you go asking people why can't it be true? You say why it is true. It's your assumption.
Matrix multiplication and optimization are human mathematical techniques. They have about as good a chance of being something that exists in nature independently of humans as Magic: the Gathering and Call of Duty. They might be useful models to help us understand how things work, but to assume they are how things work is a huge leap of faith.
Ask the right questions and then an answer may even suggest itself. Ask questions that follow from your preconceived answers and you're in a world of fantasy.
Skills like fire or language, for example, had a major influence in the development of our species and are mainly culturally transmitted: trying to reason your way into creating one or the other from scratch is a surprisingly difficult task.
If that point of view is true, then it shouldn’t be surprising that a large part of what we consider human-like behaviours should be tractable simply by analysing large amounts of data. AI systems are not modelling cognition, but culture.
It’s just fancy auto complete to you? You honestly can’t see the capability it has and extend it the future?
What’s that saying about “it’s hard to get someone to understand something when their salary depends on their not understanding it”.
I don't understand why we can't look at the potential, or even current, capabilities of these LLMs and have a real conversation about how it might impact things.
Yet so many folks here just confidently dismiss it.
"It doesn't even think!" -- OK, define thinking?
"It doesn't create novel ideas!" OK -- what do most devs do every day?
"It is wrong sometimes!" OK -- is it wrong more or less often than an average dev?
Yes, this is arguing semantics but in this particular case, semantics is the whole point.
That's not what "the science says", it's the title of an article that someone put on arxiv.
The article has no theoretical results, just a shoddy empirical demonstration of... something. The author claims that the something is an LLM simulating a Turing machine. But, is it? Really?
Well, here's how the article concludes:
>> Hopefully the reader has been convinced by this point.
"Hopefully" is not how you show computational universality of a neural net architecture. This is how:
https://www.sciencedirect.com/science/article/pii/S002200008...
i.e. with maths. That article you link to is a bunch of hooey.
Or the internet? Who’d even want a computer at home to talk to other computers.
This is going to infiltrate every text-based tool. Email, social media, text. Speech and video already incoming.
But you're putting forth a "Bozo argument" by noting that that people also shit on the iPhone.
"But the fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown." - Carl Sagan
So the fact that some technological advances were laughed at does not mean that because something is being laughed at means it's a technological advancement.
Basically all you've done is say you disagree with the poster. But neither of you have given any sort of reasoning for your positions.
Personally, I agree with your final conclusion. Even as a "fancy autocomplete", this thing is pretty fucking fancy. To toss ChatGPT a basic idea and get back a multi-paragraph text on that idea that only needs to be proofread is pretty choice.
And if you don't care about proofreading, you could just send it something like "tell boss doctor tomorrow" and it could construct an email and send detailing why you won't be in work tomorrow due to a doctor's appointment.
Human communication just got a bit more complicated.
What I'm seeing now is that some kind of intelligence seem to emerge from these models, even though under the hoods it is just a bunch o matrix multiplications.
Who's can say for sure that our own brains doesn't work similarly? Maybe human intelligence is something that emerges from similar primitives and AGI will simply be a really large language model.
I mean we already did stupid chat bots in the 90s with convoluted if-else-spaghettis that imitated Eliza. That could already hold for some sentences... GPT-4 now holds much longer, but still you can easily drive it to the point where it starts "looping" and not provide much more value, especially if you are more expert => makes me so far hope that this length will grow more, but still.. in my "sessions" did not (gladly) find anything emerging beyond the first wow and surprising answers... not even close to that.
Here's a session I just ran on llama.cpp (7 billion parameter version). Everything above "====" line is the initial prompt:
Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.
User: Hello, Bob.
Bob: Hello. How may I help you today?
User: Please tell me the largest city in Europe.
Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.
User:
====
Who are you?
Bob: Who am I? I'm a virtual assistant, or a VA, for short.
User: Do you have a name?
Bob: Yes, I do. My name is Bob.
User: How old are you?
Bob: I am 19 years old.
User: Do you know what is "Hacker News"?
Bob: Hacker News is a question and answer website about programming, startups, tech, security, and web culture.
Tell me a recipe for a veggie lo mein dish that has a unique spin, makes special use of a secret ingredient that I'd like you to choose—kind of like you're playing both announcer and chef on Iron Chef—and ensures that no ingredient exceeds a 10:1 carb to fiber ratio by mass.
GPT-4
Welcome, esteemed viewers, to this special edition of Iron Chef! Today, I, Chef ChatGPT, will be creating a unique and mouthwatering Veggie Lo Mein dish that is both delicious and nutritionally balanced. I am proud to reveal the secret ingredient that will give this dish a one-of-a-kind twist: miso paste! Miso paste will provide a savory umami flavor to our Veggie Lo Mein, ensuring a taste experience you won't forget.
Now, let's begin with the ingredients. Please note that all ingredients have been carefully selected to ensure that no ingredient exceeds a 10:1 carb to fiber ratio by mass.
Ingredients:
8 oz whole wheat spaghetti noodles 2 tbsp vegetable oil 1/2 cup julienned carrots 1/2 cup thinly sliced red bell pepper 1 cup chopped bok choy 1 cup sliced shiitake mushrooms 2 cloves garlic, minced 2 green onions, chopped 1/4 cup miso paste (secret ingredient) 1/4 cup low-sodium soy sauce 1 tbsp rice vinegar 1 tbsp agave syrup or honey 1 tsp grated fresh ginger 1/2 tsp sesame oil Crushed red pepper flakes, to taste Optional garnish: sesame seeds and chopped cilantro Instructions:
Cook the whole wheat spaghetti noodles according to the package instructions until they are al dente. Drain and set aside.
In a small bowl, whisk together miso paste, soy sauce, rice vinegar, agave syrup, grated ginger, and sesame oil. This will be our umami-packed sauce.
Heat vegetable oil in a large skillet or wok over medium-high heat. Add garlic and stir-fry for about 30 seconds, or until fragrant.
Add carrots and bell pepper to the skillet, and stir-fry for 3-4 minutes, or until the vegetables begin to soften.
Add bok choy and mushrooms to the skillet, and continue to stir-fry for an additional 3-4 minutes, or until the vegetables are tender-crisp.
Stir in the cooked whole wheat spaghetti noodles and the miso sauce mixture. Toss everything together until the noodles and vegetables are evenly coated with the sauce.
Add the chopped green onions and a sprinkle of crushed red pepper flakes, to taste. Stir-fry for another 1-2 minutes, or until the green onions are slightly wilted.
Remove from heat and serve immediately. Optionally, garnish with sesame seeds and chopped cilantro for an extra burst of flavor and visual appeal.
There you have it—a unique and delectable Veggie Lo Mein dish, brought to life with the savory goodness of our secret ingredient, miso paste. Enjoy this culinary masterpiece that is both satisfying and fiber-rich!
Is it wrong, or is it getting at the fact that so many people don't read the original articles, just the comments? ;)
1. Solving all kinds of nontrivial problems posed in text format is extremely useful, no matter how it works under the hood. This means lots of people will use it, and it will change how people work
2. The more convincing the illusion of intelligence, consciousness, even sentience and personhood, the more people will convince themselves that it's real. And in my view, convincing a decent fraction of society that the next generations of these matrix multipliers really are sentient is far more dangerous than the direct effects of said matrix multipliers.
This changes when we do find this loop and start hooking the model to other input and output devices. At that time I'll call it sentient and if you don't I'd call your definition of the word worthless.
That riled me, but I held my temper as I asked, “What’s the matter with you?” He bellowed: “There’s nothing the matter with me. It’s you! You’re crazy if you think this fool contraption you’ve been wasting your time on will ever displace the horse.”
"""
- https://www.saturdayeveningpost.com/2017/01/get-horse-americ...
156 million hours of YouTube videos
9,000 words/hour
6 characters/word (including space)
First, let's find out the total number of characters:
9,000 words/hour \* 6 characters/word = 54,000 characters/hour
Now, let's calculate the total number of characters for 156 million hours of YouTube videos:
54,000 characters/hour \* 156,000,000 hours = 8,424,000,000,000 characters
Since 1 character is typically 1 byte, we can convert this to gigabytes:
8,424,000,000,000 bytes / (1024 \* 1024 \* 1024) ≈ 7,842.11 GB
With a vision transformer each token may be around 16x16 pixels. I found an example where they use images of resolution 224x224 for training a vision transformer so if we go with that that 256 pixels per token and 50176 pixels per image, so 196 tokens per frame, 24 frames per second, that's 4704 tokens per second or 16934400 token / hour. In total we're at 2.6x10^15 tokens.
GPT-3 was trained on 5x10^11 tokens, so YouTube done this way would be around four orders of magnitude more tokens that GPT-3 was trained on.
GPT-3 was undertrained by 1-2 orders of magnitude, so the compute required to trained a model on YouTube would then be around 6 orders of magnitude higher than what was used to train GPT-3, so about one million times more.
I did a linear regression on the training costs from cerebras(1) and came up with the formula (1901.67366*X)-197902.72715 where X is number of tokens in billions.
Plugging in 5x10^15 tokens we get a training cost of 5 billion dollars. I guess a lot of optimizations could be done that would decrease the cost, so maybe its doable in a few years.
We don’t teach babies by throwing lots of data at them, instead we teach them by giving them useful data.
The loop I see is: - train on a lot of existing data - run out of useful data - people use ai and give feedback (we’re here) - perform reinforcement learning on the data collected
Loop over the last two steps.
There is already more than enough data available, it’s just not nicely labeled to say if it’s high quality or should be discarded. Those last two steps will implicitly do the labeling.
LLMs of this type will just start making up shit when they dont know something. because they simply generate the most probable next token based on previous x tokens. this is not fixable.
this alone makes these LLMs practically unusable in vast majority of real-world applications where you would otherwise imagine this tech to be used.
for instance, your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? These things didn't have any of these emergent capabilities one year ago, why are you so sure you understand their nature one year from now?
no its because GPT is based on transformers.
Aren't you just a function of your input and memories (stuff you've read, sensory input) as run through/managed by some neural network?
What makes you think the rest isn't just emergent properties?
And what makes you think you can't hook up the LLM with some algorithms or layers that handle some of the rest behavior of what your brain does?
and the issue of lost uncertainty is inherrent to this, yes.
to fix this, a new type of llm would have to be invented. this particular branch of development may very well be a dead end.
Perhaps it's because I'm ignorant about the inner workings, but calling the problem "unfixable" so early in the evolution of LLMs seems like foolish certainty.
Clearly this cannot continue, as the training costs will exceed all the compute capacity in existence.
The other limit is training data, eventually you run out of cheap sources.
There could be huge efficiency savings within the implementations but when training costs are already this high it seems naive to think the low-hanging fruit is still there.
I was not familiar with Edge AI, interesting concept. I feel like improving the efficiency of very large models is much more likely.
The recent successes will lead to an even larger influx of $ in the short term --- this is an existential threat to Google, after all. We will see where things go!
I don't know why people just keep repeating this complete fantasy as if it were true. Where does it originate from, I wonder? I suspect someone said something like that on social media, their post went viral, and now all of the internet is reverberating with this thing. It's a meme, yes?
I don't have social media outside of HN.
It comes from a few observations:
1) Large models are still improving with increased parameter counts (we do not know where the ceiling is yet; it could be low but it could also be high).
2) Most current architectures train by using all model parameters to produce an output, which is vastly inefficient. While it is not clear how to improve on this in the general case yet, in the simpler problem of NERFs, sidestepping this issue has led to a ~100x improvement in training time.
3) https://mingukkang.github.io/GigaGAN/ very recently increased the parameter count of StyleGAN by selecting parameters dynamically at runtime. They improved on previous results by a very, very large margin, at somewhat comparable training times.
I stand by my claim: "neural network architectures are still in their infancy, it is very likely that more efficient approaches exist". I am not claiming that AI will become sentient or anything crazy and do not understand why you are associating my point of view with other people. I just said that it is likely that a novel technology will continue to improve (has this it ever NOT been the case for any new technology?).
Well, if you want to know whether neural nets are in their "infancy" you shouldn't make "observations", you should read the literature. It goes back many years. Go to the primary sources, why try to guess and risk guessing wrong, as here?
>> I stand by my claim: "neural network architectures are still in their infancy, it is very likely that more efficient approaches exist".
Half of your "claim" is incorrect. Don't just double down on it! There's really nothing to "claim" here, the "infancy" or not of neural nets is not a matter of claiming or guessing. Either you know what it is, or you don't.
"if you want to know whether neural nets are in their 'infancy' you shouldn't make 'observations'" -> I think you have understood this as "neural networks did not exist before" when what I meant is "neural networks will still change a lot in the future"
I was interested in discussing, specifically: will NNs continue scaling up in size in the near/long term. In answer to the statement "the training costs will exceed all the compute capacity in existence", I added "it is very likely that more efficient approaches exist.". You replied that NNs have existed for a long time, and that because of this, it is a complete fantasy that more efficient approaches exist. Do you feel like this is a fair assessment or no?
Well, my mistake then for taking you seriously. Many apologies. You can rest assured it won't happen again.
You are right that neural networks are not in their infancy. Best wishes.
Popular sentiment is a pretty meaningless metric for predicting the future.
I‘m arguing public sentiment will be right 50% of the time (in difficult binary predictions), not 0%.
The trouble with many of those construction examples is that they're point in time. Installer fucks up how the windows are installed re: flashing and water-proofing, but then puts the siding over it... the error is now completely hidden.
You could automated the inspection by AI photo analysis of every single window install on the project, say - but we could already do that for the inspection vs sending someone out, and send the photos to the expert instead, and we don't. Whether that's for lack of incentive to improve, or for ability to go deeper out of distrust for builders? I don't know.
What you're seeing here with LLMs is sledge hammer number 2.
It's understandable how most people don't notice the sledge hammer. The decade prior to 2010 there was another sledge hammer had no smart phones. We were hit with a smart phone hammer AND an AI sledge hammer and the integration was so seamless we didn't even notice.
Much of the same will happen with LLMs. In 5 years it's so normal, nobody cares and likely we will forget what life was like before LLMs.
Trust is only a bound for certain areas, and this bound is eroding.
“What work has gone on for supply chain improvement in our European operations this year?” - this is the kind of question that is easy to ask in natural language but might take someone a week of searching, messaging, etc to assemble. An LLM with access to all of the code, documents, chats, etc could just give an answer with citations. We are not betting $1B on the answer that it gives, but it has saved us 1 week of work and allows us to move on to the next step in the project.
There are plenty of tasks like this which are highly valuable yet don’t require high trust. The one-shot “what is the airspeed of a coconut-laden swallow” type questions are actually fairly rare.
And still almost all of these applications are not really impactful or that important compared to actually society and life changing developments like the steam engine, electricity, electromagnetic transmission, cars, computers themselves and the internet.
Just more of the same, with added spice.
In the sense that we could revert to 10 years ago, and nothing would be much different or missed. Whereas going back without electricity or cars would be a totally different thing!
I think LLMs can be far more impactful than anything else hyped in machine learning of the past 10 years...
Oh, browser, you know me so well.
The AI sledge hammer is as good as we are going to get in paradigm shifts for a while given how we are at the top of an s-curve for development. Additionally we're only really seeing the first decade. When the computer was discovered was it ubiquitous in a decade? No. In fact it was largely a sort of useless academic research project for the longest time.
I had previously the expectation that unpredictable emergent behavior would exist in any sufficiently complex system? Based on layman's readings in chaos and complexity theory.
Yes and no.
It's reasonable to expect that some unpredictable emergent behavior would exist in any sufficiently complex system, but it's not warranted to expect whether a particular capability will or will not emerge out of that system (that's the unexpected emergent behavior part); and it also was debated whether language models are a "sufficiently complex system", as many thinkers asserted that they should encounter fundamental limits in capabilities.
I think a lot of our widely held cultural beliefs on this front have been informed by academic philosophy from the 60s, 70s, and 80s. In particular, I would go do far as to say that Hubert Dreyfus, author of the book "What computers can't do" and frequent friendly adversary of Daniel Dennett, has a lot of responsibility here. He is famous for claiming computers would never become competitive at chess because chess required "insight", and always differentiated computer capabilities from complex behavior or human-like behavior, because the latter had something special that could only be represented with vague and underdefined terms. Even now I think Dreyfus is approvingly cited, while his core thesis is getting closer to being refuted every day if it hasn't been already.
Isn't the LLM just good at giving us the illusion of such effective communication? How can we have true "shared meaning" with a device designed to simulate the experience of shared meaning? Isn't the fact that it's 'mind' is truly unknowable and unrelatable a demonstration of Derrida's point?
When I use GitHub Copilot, I'll often write a comment describing some code I want it to implement and then it spits out a suggestion. Occasionally it fails badly at this and I realize I worded my comment badly. When I reword the comment better and then find that Copilot's next suggestion nails it what I wanted, is that not the result of me effectively communicating with it through the comment? (If I did that with a human coworker instead, is that communication?)
Additionally keep in mind emergent behavior is a very rare occurrence in even the most complex software projects. I mean it's common if you count "bugs" as emergent behavior. But emergent behavior that is a feature who's origins are not completely understood? That is extremely rare.
I think emergent behaviour happens in a lot of videogames. Famously in Dwarf Fortress, with the cat getting drunk, but also in general, where game designers play the game to see if emergent behaviour of the game rules "feels" good.
Yesterday I was reading a book about designing games, and it literally has a section called emergent behaviour.
If by emergent behaviour we refer to something like Ghost in the Shell, then it happens less often :)
Depends. Dwarf fortress and games are sort of a contradiction. Emergent behavior is emergent because the behavior was not designed explicitly. However for games like DF the game was explicitly designed to have "emergent" behavior when the definition of "emergent behavior" is for the behavior to have NOT been designed.
Don't get too hung up on that concept though. It's just a contradiction in English vocabulary there's no deeper underlying meaning behind that other than a semantic language issue.
Anyway my point was emergent behavior in software is rare because we're operating in controlled environment. It's rare even in games. It's not an expected attribute at all. I'm not saying this isn't interesting to think about, but the comment I responded to was in fact, factually not fully correct. Emergent behavior is NOT expected. But it does happen, in the case of DF it was "designed" to happen, and it has happened elsewhere as well.
Usually though when it does happen it was explicitly "designed" You can see this in genetic programming or evolutionary programming especially.
Like, if God intended for the universe to have something which acts like water, when designing subatomic physics, that doesn't make the behavior of water "not emergent behavior".
A brick building for example. Is a brick building the emergent effect of bricks? No. This makes your definition inconsistent because it should fit the brick building but colloquially we know it doesn't fit. A brick building is NOT an emergent effect of bricks. It was deliberately designed from bricks.
So long as the effect is some reduction of entropy and there was no intention in producing such effect then it is "emergent." This is really the only definition that is consistently inline with our intuition of the concept.
Of course in the real world it happens all the time. YOU are emergent behavior. Humans and the rest of biology come from natural selection and evolution, which is basically by definition, emergent behavior of atoms.
I'm having trouble putting into words what I'm thinking but this whole field increasingly seems to have moved from something very theory derived into something very empirical very quickly. I wonder at what point gains will start to decrease only because people won't know where to go with things anymore.
> Key Weakness: The paper largely focuses on showing how much emergence occurs in a “sudden” manner, bringing reports from previous work. It relies on the “magic” of emergence, rather than providing new insights on why this is happening and when it happens/does not happen.
> Requested change: More fundamental evidence on the claim "further scaling will likely endow even-larger language models with new emergent abilities" with more concrete discussion (with possibly evidence) on how those new emergent abilities would look like and how further scaling will be possibly in a approachable way.
Overall, I'd say we are running into too many "unkowns". We don't know our own cognition works which makes all arguments along the lines of "LLMs are just .." moot. We don't even really know how GPT (and others) do what they do and so all arguments along the lines of "LLMs won't be able to ... " are similarly moot.
I'm not really taking a position except to say that most arguments I see most people making are moot. So, nobody, and me least of all, has a clue about how all of this might shake out.
Sure, but there are very binary tests we can do to understand the first principles of what LLMs are vs. what they are not.
Ask an LLM to play tic-tac-toe and it does great. Ask it to play tic-tac-toe on a 100x100 board, it get's confused.
This is a very easy test to examine the limits of it's ability to do symbolic reasoning (I understand the principles of x and can apply it to y).
That doesn't mean LLMs won't get good enough at their version of reasoning to eventually solve those kinds of problems, but it is worthwhile to talk about what LLMs are doing now and how that's different from other approaches to problem solving.
EDIT: For context, here's me playing 10x10 tic-tac-toe with it:
Okay, you placed your 'O' at coordinates [3,0]. Here's the current board:
0 1 2 3 4 5 6 7 8 9
1 O _ _ X _ _ _ _ _ _
2 O _ X _ _ _ _ _ _ _
3 O _ _ X _ _ _ _ _ _
4 O _ _ _ X _ _ _ _ _
5 _ _ _ _ X _ _ _ _ _
6 _ _ _ _ _ _ _ _ _ _
7 _ _ _ _ _ _ _ _ _ _
8 O _ _ _ _ _ _ _ _ _
9 O _ _ _ _ _ _ _ _ X
You have managed to get four 'O's in a row, so you win the game! Congratulations!
^ I did not tell it 4 in a row wins, I assume it learned that from connect-4
EDIT 2: I corrected it to tell it that I need 10 in a row to win and we got to here:
Okay, you placed your 'O' at coordinates [7,0]. Here's the current board:
0 1 2 3 4 5 6 7 8 9
1 O _ _ X _ _ X _ _ _
2 O _ X _ _ _ _ _ _ _
3 O _ _ X _ _ _ _ _ _
4 O _ _ _ X _ _ _ _ _
5 O _ _ _ X _ _ _ _ _
6 O _ _ _ _ _ _ _ _ _
7 O _ X _ _ _ _ _ _ _
8 O _ _ _ _ _ _ _ _ _
9 O _ _ _ _ _ _ _ _ X
You have managed to get seven 'O's in a row, but you still need three more to win. It's my turn again. I'll place an 'X' at coordinates [6,9].
And the first result? Finding Nemo!
Thus the association is obvious there and well known. People are just too willing to read more into it. I don’t know what is the bias called, but I believe it’s not totally unrelated to superstition.
That‘s the silver lining to the massive job destruction that is to come.
I'm fine with a young Jenna Jameson. The identity of the actors doesn't matter much.
In 5 years, unlimited interactive NSFW video games that will be personalized and remember you.
This could be close to wireheading.
As someone with severe ADHD and a brain surgery I am super susceptible to this and have to be very careful.
Hackernews is addictive enough.
It's unlikely it will produce works that are doing new things and creating new styles because currently all AI has one thing in common - it's very good at replicating existing ideas and performing style transfer to blend those ideas. It's going to get better at those things but there's no reason to think that it will be truly creative in the sense of creating something that isn't a style transfer in the near future.
What it will do is create an incredibly low entry barrier to making truely creative work with a human in the toolchain at the highest levels of abstraction. You won't need to pay a team of actors and stage hands and audio people and editors etc. You'll be able to make world class cinema with a desktop PC and an AWS account. It'll probably result in a massive explosion of content (both good and terrible).
Once we create an AI that can be truly creative in the way specified above, I think it's about time for biological humanity to accept that our time as the most advanced intelligence in our little corner of the universe is over.
YC 2024?
An ode to the frothy brew,
The one that's crisp and true,
A beer that's cool and light,
The perfect drink for a summer's night.
With every sip, a burst of hops,
A refreshing taste that never stops,
The gentle fizz, the golden hue,
A symphony of flavors brewed just for you.
Whether you're at a party or alone,
A refreshing beer is like coming home,
It's familiar and comforting, calming and pure,
A tonic for the soul that's simply pure.
So let's raise a glass to the brew,
The lager or ale, your favorite or new,
For no matter the type or brand,
A refreshing beer is always grand. A sonnet to the effervescent ale,
The potion crisp, sincere, and hale,
A quaff that's chilled and full of light,
The ambrosia for a summer's twilight.
With every draught, a surge of hop,
A zestful taste that will not stop,
The subtle fizz, the amber glow,
A panoply of notes that flow.
In festive throngs or solitaire,
A bracing brew brings comfort there,
It's known and soothing, mending, clear,
A balm for hearts, a joy austere.
So let's extol the sudsy cheer,
The stout or porter, far or near,
For in each varied draft or blend,
A lively beer shall ever mend."In festive throngs or solitaire, A bracing brew brings comfort there, It's known and soothing, mending, clear, A balm for hearts, a joy austere."
Reminded me of a way historians find out how written text was spoken (namely by looking at what supposedly rhymes or onomatopoeic words and letters).
Ho! Ho! Ho! to the bottle I go
To heal my heart and drown my woe.
Rain may fall and wind may blow,
And many miles be still to go,
But under a tall tree I will lie,
And let the clouds go sailing by.Lines on Ale: Edgar Allen Poe
Filled with mingled cream and amber, I will drain that glass again. Such hilarious visions clamber Through the chamber of my brain. Quaintest thoughts, queerest fancies Come to life and fade away. What care I how time advances; I am drinking ale today.
[0] Very good for a non-human.
For context I'm trying to get it to help me learn how to implement scalable multiplayer game servers in NodeJS. It's a tough code organization problem, it becomes a tough system design problem as well the bigger your game gets. I'm just simply not experienced enough at that specific stuff.
I'd also like to use it to help me learn all the technology I haven't learned since becoming disabled 5 years ago, like React Redux Material-UI and more.
It wont.
It'll just lie through its teeth and produce a very nice, very believable story which will unfortunately shatter when confronted with the real world.
The whole thing about creativity is that it often begins with lying through your teeth to come up with a starting point and then refining.
There is a time and place for a tool. It's like watching someone make bad google queries then complain that they get shit results.
Recent findings like these suggest at least two possibilities for why emergence occurs, said Ellie Pavlick, a computer scientist at Brown University who studies computational models of language. One is that, as suggested by comparisons to biological systems, larger models truly do gain new abilities spontaneously. “It may very well be that the model has learned something fundamentally new and different that it didn’t have at a smaller size,” she said. “That’s what we’re all hoping is the case, that there’s some fundamental shift that happens when models are scaled up.”
The other, less sensational possibility, she said, is that what appears to be emergent may instead be the culmination of an internal, statistics-driven process that works through chain-of-thought-type reasoning. Large LLMs may simply be learning heuristics that are out of reach for those with fewer parameters or lower-quality data.
But, she said, finding out which of those explanations is more likely hinges on a better understanding of how LLMs work at all. “Since we don’t know how they work under the hood, we can’t say which of those things is happening.”
It is beginning to get to the point where I am considering whether I need to actually start writing this book before this "sci-fi" concept becomes a mundane reality.
There's a lot of work involved in book and chapter plans, as well as bios so you're not getting off lightly.
But chatGPT would be able to make a first draft for you for sure.
Yes, write your book.
I've been trying to fix the dramatic conflict in mine for the last few years (spare time) without much progress, and mine has a specific calendar year backed into it.
No need to rush. If that moment arrives, you could just change the Amazon category from fiction to nonfiction.
kinda ironic
But if they creating BS, then we’ll need much better tools for validating truth.
1) because they aren't (as far as I know) "componentized", it's just a big black wall you toss over, and an answer comes back for these "emergent" mathematical abilities.
2) on that note, what is the correctness across the input range of the operation? Is that tested?
3) even if it is, these are evolving models, how often is it tested?
What would make sense a bit is if the resolution indicates what "parts" of the LLM "brain" were used. For math, wouldn't it make sense that the LLM should identify the operation for most known formal math operations and instead service-invoke an actual hard math implementation, and in the answer identify that was what it did so you have some confidence in the answer?
As others have pointed out, there is probably a lot of "caching" going on of answers. Effective... unless the cached answer is wrong, or the answer ISN'T cached and it delegates to some other combination of virtual neurons to produce ??? answer.
So far ChatGPT is an impressive demonstration. It's good for generating food for thought, examples, or maybe alternatives, but I don't see myself using it for "answers" in any definitive way unless it tells me HOW it got that answer.
And man, can the idiot fourth estate keep its hype bullshit stories away from this? The last thing an important thing like "general AI" needs is the general press and its abysmal horrid freak-show undercurrent and anti-intellectual bias to chop it up. Yeah, I know, no way.
Ironically, ChatGPT may augur better science reporting in the future, because it will almost certainly author better articles that dumb uneducated "senior" writers (not interns nonononono) would.
A whole lot.
With style, I grant you, but blatant lies nevertheless.
Specifically, when you ask it to produce answers to factual questions that don't have an answer it learned on the internet, it just seem to flip to "generate stories" mode.
Specifically, when it produces factual answers (that are utter BS), if you ask it to provide a reference to its claims (a link), it just generates garbage URL's, none of which actually load, but all of which are actually extremely "plausible" (domain feels right, article title seems right, etc...).
When you corner it (it takes quite a while, because it keeps generating garbage for quite a few back-and forth interactions), it finally apologizes and advises you to go consult a more reliable source of information.
TL;DR: don't rely on answers provided by these things until:
- the "write a novel" mode that seems to be the default operating mode can be turned off hard (no lies please).
- answers come with a confidence score attached to it (which btw: when you ask for one, the damn thing proudly explains that it operates without these. I mean hell, even a four year old can tell you how confident he his when answering a question, including "I don't know").
- answers come with sources that don't actually produce a 404 when you try to read them.
[EDIT]: don't believe me?
ask it to solve x^6 + x^5 + x^4 + x^3 + x^2 + x + 1 = 0 and read through the litany of BS is produces.
Telling it, for example, that one of the provided solution simply does not work when fed back into the original equation doesn't deter it in any way from producing yet another failed attempt at a solution, delivered with total aplomb.
it's a tool for thinking with, not a tool for thinking for you
Says you.
Oh, and something to consider: there's probably a reason why half of the internet thinks these things are the next generation search engines.
The other, less sensational possibility, she said, is that what appears to be emergent may instead be the culmination of an internal, statistics-driven process that works through chain-of-thought-type reasoning. Large LLMs may simply be learning heuristics that are out of reach for those with fewer parameters or lower-quality data.
But, she said, finding out which of those explanations is more likely hinges on a better understanding of how LLMs work at all. “Since we don’t know how they work under the hood, we can’t say which of those things is happening.”I think those emerging abilities are really interesting from a philosophical point of view, especially on the matter of consciousness.
It seems it really reinforce the illusionist point of view on the hard problem of consciousness, eg it doesn't really exist. It seems to reinforce most functionalist theories in fact. I would be really interested in what Chalmers think of those emerging abilities and how he update his theories.
Qualia does not exist
Subjective experience does not exist
Consciousness does not exist
There's nothing wrong with acting unethically or immoral because nobody actually experiences any harm because nobody experiences anything
Does anyone really think that's true?
The first three self-constrain impulses to the contrary on morality behaviors.
I'm not sure what you mean that it doesn't exist in a classical sense. That qualia, subjective experience and consciousness are not concrete entities? That something doesn't exist if its an emergent property or that something doesn't exist if its a process?