Meta AI Unleashes Megabyte, a Scalable Model Architecture
artisana.ai
artisana.ai
MEGABYTE is indeed a cool new architecture, but it is still very much just a proof of concept at the moment. The paper shows that the model can compete with (but not decimate) vanilla Transformers on the scale of ~1B parameters in long-sequence prediction tasks. They did not "unleash" anything, and scalability to very large parameter counts and datasets still has not been tested.
I'm certainly very excited to see where this architecture goes as the community gets ahold of it and starts developing it, but to call it "revolutionary" this early on is disingenuous. I personally have a few experiments I want to run with it, but I put the probability of it being a true GPT-killer at <20%. I would love to be wrong, though!
I've added links to both in my bio.
Let me just pretend this word isn't there, it's probably an exaggeration.
It takes a special kind of adaptation to filter out all the 10%-40% bluff usually found in these kinds of news articles.
https://chrome.google.com/webstore/detail/actually-no-remove...
Maybe I should extend it :)
Thanks for the actual sober analysis. As not-an-AI-researcher, I could have been easily bamboozled by an article like this.
One thing to keep in mind in particular is that the vast majority of transformer alternatives/improvements/optimizations that initially showed promise ultimately ended up scaling less well than the baseline transformer architecture. The transformer has been weirdly hard to improve/dethrone.
This is part of why I’m not worried about the current crop of generative AI. I am however both curious and concerned about what the tsunami of talent and $$$ chasing the current trend will achieve.
If this n^(4/3) alt transformer compute scaling is real (and there’s been many a pretender, so it’s too early to tell), then that could fundamentally change the overall AI scaling law, substantially lowering the brick wall.
And that could be a game changer.
Think of a human with below average intelligence. Then think of a human genius. Now consider how incredibly similar their brains are, despite the massive performance gap. It's not like one has 10x the number of neurons/synapses/connections etc. of the other. They're both healthy human brains, and you need powerful technology to even distinguish them structurally.
Considering this, it seems perfectly possible that a model like GPT-4 is just a hair's breadth away from vastly superhuman performance. It certainly beats the average human at many tasks already. The gap between a moron and a genius is a lot larger than that between GPT-4 and a superhuman.
This is not the revolution you think it is.
```python > 338383*887282 ... ANSWER ... ```
Yet skynet never came.
LLMs are not AGI. A human with below average intelligence is still a league above a chimpanzee. A chimpanzee will never be able to read, not because "it's too dumb", but because a chimp's brain lacks the actual hardware for reading. The LLM is the chimpanzee. The gap between an LLM and a "human with below average intelligence" is far more than 10x.
But asking for a complete application nearly always fails, when an average human that knows programming should be able to make one that works.
"This talking dog is a dumbass. The risotto recipe he gave me sucked, and the C++ code he wrote is full of security holes. I don't see what all the hype is about."
Hint: ML will get better. Humans will not. ("Slow moving target," indeed.) What we're learning is just how many aspects of our vaunted 'intelligence' are really just features of our languages and the graphs they encode.
GPT-3 knows most programming languages better than I do, even though I literally learned to read with the Commodore 64 user manual back in the 80s and haven't stopped being a nerd since; and while GPT code isn't always correct or even compilable, it's not like I don't still make mistakes that cause compilation to fail a few times a day, and there's a reason we all insist on testing code rather than just assuming it will work when a dev stops typing.
Again, I acknowledge GPT-3.5 (and I assume 4 but have not used it) is not perfect: like others I'd call 3.5 a "junior developer" (and it isn't even that good in every subject!); yet, despite that, the only domains where I can regularly beat it at those where my perception of reality is fundamentally different from its perception (how words sound and how numbers are composed, so it's relatively bad at arithmetic and rhyme), or where it has been forced to be bad (ask it to write about conflict, my experience is everyone reconciles and lives happily ever after).
But in most cases it would take me years to get as good as it already is. And there are more of those subjects than I can remember the names of, too.
Very few people who "know programming" are actually capable of creating a complete application that serves a specific purpose. Many professional programmers struggle to implement basic algorithms like prime number testing. Coding AIs absolutely outperform the average programming professional, because the average programming professional can barely program. The software industry's demand for programmers is far too great for every coding position to be filled with top-class full stack engineers.
There's a lot of things you did during university classes. I bet you don't even remember half of them, and of the half you do, you couldn't actually do most of them from memory right now.
The advantage LLMs have over programmers is that they've seen much more code than any human ever would, remember pretty much all of it - not necessarily verbatim, but also as learned concepts, and that this knowledge does not decay. It's all equally accessible. Whether you ask LLM for basic or optimized primality testing algorithms, or how to use some obscure part of Microsoft's DCOM that approximately nobody on the planet wrote new code for in the last decade - they'll perform equally well. Not perfect, but not worse than an average software dev that does remember the things they're asked to work with.
Yes, and LLMs show all of the specific things you've listed.
https://github.com/drifting-in-space/botsh
Check out some agents that probably can already do more than you knew about.
In which direction?
GPT-4 passes the bar exam with a top 10% score. How do you think a human with below average intelligence (or even with average intelligence) would fare?
Copilot generates programming code that solves problems, and in most cases the code is correct. It outperforms many junior professional developers. Do you think a human with below average intelligence could do that?
Absolutely, it's why Javascript is so popular.
Do you mean a programmer with below average intelligence (for a programmer)? Because I'm having a hard time believing that you actually believe what you wrote.
BTW, JavaScript is actually a fairly difficult language to learn and use. Python, BASIC, and even Fortran are much simpler conceptually and have far fewer pitfalls. JavaScript is popular because it's the only language that every modern computer has an interpreter for, not because it's so easy that idiots can use it.
I don't understand what rote memorization to pass a test has to do with intelligence. For the record Kim Kardashian passed the bar; I imagine anyone given the proper motivation and time to study could do it, it's not a hard test.
If AGI is a computer passing a test, then AGI was achieved a long time ago. I don't think a human with below average intelligence could multiple 2 very large primes but I don't mistake my calculator for intelligence. A below average intelligence human can drive a car with a couple hours of training, an LLM can't do that (with a far lower power budget as well).
I'm not saying AGI is impossible, but it's clear LLMs are not AGIs; it's not a matter of having a 100x more powerful LLM, just like making an Ape better at sign language won't make them better at abstract reasoning. An Ape's brain fundamentally lacks the mental machinery for higher level things that humans do. It's not a question of not being smart enough. LLMs are simply one component of the human mind.
AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing.
> it's clear LLMs are not AGIs
And the main argument for that is that "it's clear". They're beating lawyers, doctors, and software engineers, but obviously, that's not real intelligence...
Yes! Captchas so effective!
For instance, if it could make a peanut butter and jelly sandwich? Most challenging things that are easy for us are in the motor domain. While important, I think “intellectual AGI” is a meaningful milestone and closest to what most people think of when they think AGI.
The "G" in AGI is general. A computer program or system that could both, lets say write code and learn drive a car would be something closer to an AGI. Written tests are remarkably brittle in showing how intelligent someone is - like we already know the limitations of tests in the real world! Einstein famously flunked his entrance exam, but then invented general relativity at 26; but other posters would have you believe an LLM is more intelligent than Einstein because it could pass a test. When an LLM defends a dissertation in arguably any field that would be way more impressive than an LLM passing a test that humans already designed and know the answers to.
One problem I have with saying a 100x more powerful LLM could become AGI is that there is nothing that leads me to believe that LLMs, as they exist currently, are capable of synthesizing new knowledge and I'm not sure what breakthroughs you would need to get there. Once you start to think about that, you start to run up on the limitations of the LLM. If I were to invent a completely new programming language, I could probably teach a junior engineer how to use it in a week, but the jury is out if I would need to first generate 50,000 sample program and spend $1,000,000 in gpu compute to get an LLM to output the same thing. It's hard to consider such a system AGI. Further still, sure you have Google spending millions of FSD, but it's hard to consider the system they have as general. Could I take Waymo and have it pilot a forklift? Or a submarine? How much would that cost? A """below average intelligence""" human could learn to use a forklift in an afternoon.
All in all, there's more to intelligence than written tests.
This is a very problematic word. For example if you were a civil engineer and went "throw me any old design for a bridge, I have a river I need to cross", you'd have your license removed.
Intelligence is too massively loaded, and too much of a gradient even across humans to try to some up human or AI abilities. It is a multitude of different capabilities that don't necessarily have to be bundled together for something to be 'smart', 'useful', 'capable', and/or 'dangerous'.
> AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing.
You're narrowly defining what a "test" is. And therefore the "until 5 years ago" doesn't make any sense. A test is not just written exams like a "third-grade reading comprehension exam." Is playing chess against the world champion not a test? A trad AI computer program beat him 25 years ago, not 5 years ago. Is diagnosing patients with a particular disease better than the average untrained human not a test? An expert system did that 50 years ago.
Your point is not a refutation of the parent post.
I think we all can agree that even the best LLM currently is not AGI. That's not what being disputed here I think.
However a 100x more powerful LLM is not just 100x better at recall. A 100x more powerful LLM is not just 100x better at being stupid hallucinatory parrot. A model that is just 100x bigger is not necessarily 100x more powerful if you define power is the ability to achieve goals.
However pure language models will always lack something else: the ability to ground things in reality.
I recently had a dream where I solved some problems and when I woke up I realized that those solutions were bullshit, but I also realized the whole approach of my dreaming self was very similar to what a LLM would have done.
We only have our 5 senses to go off of. Meta has already put out one multimodal model incorporating multiple data types, openai is undoubtedly working on it too.
It's not a matter of which senses you have, but about being able to "continuously" use them.
The current LLMs are basically unfiltered raw thoughts that must be continuously refined. A similar thing happens in our brains and only a little bit of that is accessible to our consciousness
Exactly. But, AFAIK, it's also the part that does the bulk of actual thinking and decision-making for us. In that sense, LLMs may be closer to AGI than people expect, because they seem to be capturing the actual core of intelligence and reasoning - and the missing bits (like long-term memory and higher-level thought stream filter/censor) may be much easier to bolt on to them.
The output of LLMs is kind of like our stream of consciousness, there's a lot of things I think, then discount after internally reflecting on the thought which the often leads to a more correct solution. Having an LLM 'think' like this natively would massively increase the necessary the amount of compute needed, hence the expense, so at least in any public products it's not being done at this time.
I wish I had time to play with it some more right now. The pace of progress in the field is giving me a serious case of FOMO.
Balance, proprioception, hunger, …
Disagree, for the record. If I’d described the capabilities of contemporary AI to 100 AI scientists 5 years ago, I bet more than half would agree to call that AGI. Further, more than 90% would assume that these capabilities were decades and decades away.
This is hard to believe, the all you need is attention paper was 6 years ago, GPT1 is 5 years old and GPT3 is 3 years old. The current crop of LLMs wasn't something that happened overnight.
While I know the wider point you're trying to make, Kim Kardashian is a very smart business person. She may just not fit your narrow definition of "intelligent".
I believe you do have a good point overall, but this here is, IMHO, a rather bad example, because I'd argue GPT-4 is already capable of abstract reasoning, and this seems to be exactly the skill that improves with increasing dimensionality of the latent space.
I agree that LLMs are equivalent of a single component of a human mind. Specifically, I think they're closest equivalent to our inner voice / inner thoughts. But this part is arguably exactly the one that does most of the abstract reasoning (and most reasoning in general), so I think in fact LLMs do have the "hardware" for that specific aspect. What's lacking right now is the equivalent of the higher, "conscious" layer, that guides, filters and censors the stream of thought. That, and long-term recall. Short-term memory might atually correspond to what the context window is in LLMs.
That, and fusing in all the other senses (sight, sound, smell, taste, touch, time, etc.).
The reason scientific researchers in this area are using the term 'AGI' so much is that it does fit the definition of AGI ... *For some definition of AGI*. And there lies the problem - no one can really come to a good consensus on a good definition of AGI. This is why many scientists in this area are avoiding the question altogether - the question is loaded, and is misinterpreted by the public if statements are made.
So, for example, if I make the statement here that e.g. GPT-4 has intelligence which is general (AGI), it will likely be met with a rabid response from HN. However, the claim may be more dull than you're expecting. People often conflate AGI with things that are not required, such as agency, etc.
This definition from [journal Intelligence Vol 24, No1, 1997] can be that "some definition" of AGI: must be able to 1) think abstractly, 2) comprehend complex ideas, 3) reason, 4) plan, 5) solve problems, 6) learn quickly from experience.
Many of these GPT-4 can do, if some modifiers are allowed - for example, GPT-4 can learn quickly from experience so long as you aren't starting a 'new' GPT-4 system from scratch every time you want to interact with it. This is probably preferable, since much of the experiences it will have are personal to the individual working with it, and it would be highly undesirable to do the opposite here. Planning was difficult for the system early on, but it has appeared to learn that it is helpful to lay out a plan early on in a large task, so that appears to be a capability as well, at least on a basic level qualitatively. There is some good literature on ability to reason (and solve problems in abstract and complex ideas) from Microsoft's group on causal reasoning. In pretty much all tasks of causal reasoning the system can achieve near human performance, and LLMs as a category outperform previous SoTA from more targeted or specific systems made for causal reasoning.
Anyway, I would suggest to anyone that has strong reactions to claims about AGI to realize that they are likely building up the statement to be more than it is. Perhaps similar to 'machine learning' may have been misinterpreted years ago ("A machine can learn?! There is no tomorrow!!"), what is being stated here is often more narrow than you may believe.
Turns out that may be as much marketing as truth.
According to this paper[1]:
"although GPT-4's UBE score nears the 90th percentile when examining approximate conversions from February administrations of the Illinois Bar Exam, these estimates are heavily skewed towards repeat test-takers who failed the July administration and score significantly lower than the general test-taking population. Second, data from a recent July administration of the same exam suggests GPT-4's overall UBE percentile was ~68th percentile, and ~48th percentile on essays. Third, examining official NCBE data and using several conservative statistical assumptions, GPT-4's performance against first-time test takers is estimated to be ~63rd percentile, including ~41st percentile on essays. Fourth, when examining only those who passed the exam (i.e. licensed or license-pending attorneys), GPT-4's performance is estimated to drop to ~48th percentile overall, and ~15th percentile on essays."
[1] - https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4441311
There was an article on HN yesterday debunking that. OpenAI is prone to exaggerating
Now, can and should all structure necessarily come from pure training? Is there a better "seed" structure that can make models more effective? Is there some other bits that are missing that are not in the microstructure but in the way the model is connected to the external world and itself (feedback loops etc)?
It seems to me that LLMs are perhaps capable of making up the inner thought loop, and the rest of the obvious systems seem doable. The context seems to be the hard problem; pulling in the proper things, and giving them the correct amount of attention.
This is likely one of the harder problems to solve. One of the strengths humans seem to have is the power of analogy. These analogies quite often can lead us into new paths of thought, or at least in the correct direction.
How do you do this in a machine system without wasting tons of power chasing dead ends, not sure?
Scaling can happen in two dimensions - model size and dataset size. What counts is the product of n_examples x n_parameters. That's why we have the super-Chinchilla laws, where n_examples >> 20*n_parameters. Scale the data, keep the model lean. Not to mention dataset quality - if you got clean diverse data you need less of it.
Another important trend is using LLMs to generate training sets. Example: ConstitutionalAI (pure RLAIF), TinyStories (scaling down LLMs), Alpaca (borrowed RLHF), AlpacaFarm - a recent paper promising fine-tuned models for $200 cost in 24h.
Tried to explain here: https://news.ycombinator.com/item?id=36054809
I don't think you are givin these "below average" intelligence individuals enough credit. What we consider a genius is the equivalent of a dog show obstacle course. We measure intelligence/genius as whatever is hard for humans and completely ignore what is easy because we fail to see the complexity behind the easy stuff.
Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc).
Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don't use any methods to question training input/evaluate it against current knowledge etc. Our current language models are based on reciting/memorization/force-feeding, not true learning.
Computer models have massive underlying advantage of working on CPU/GPUs where they can be modeled, cloned, retrained, have binary accuracy, can integrate with specialized code instantly, have access to massive memory storage, they are insanely fast and precise etc.
Looking at it from first principles, there is no reason to think that near optimal runtime should not be more efficient than human brain on currently available computers.
We don't need to simulate full brain just like we didn't have to create super fancy legs to go to the Moon.
Wheels are trivial but useless without bearings.
Bearings most certainly aren't trivial. All the things you listed further are dependent on bearings somewhere.
Evolution is a greedy, lazy optimizer, so it promotes things that work a-ok for a given environment.
It's also worth noting that wheels alone are not too useful for transportation, as they're only half of the picture. The other half is roads. That is, because we couldn't (and mostly still can't) figure out all-terrain mobility systems that could navigate diverse environments, we cheated and locally flattened the environment, to reduce the problem to that solvable by a humble wheel. Evolution can't cheat like this.
I can imagine evolution creating macro-scale wheels that can be healed, and/or able to survive long enough before failing to be advantageous - after all, bones and teeth can hardly be healed if badly damaged, and yet they last long enough to stick around as core design elements.
This is why I mentioned roads. Whether or not evolution could iterate its way to macroscale wheels, it wouldn't, because they'd be useless without roads. Legs may be more complex overall, but they're an all-terrain solution that can be incrementally improved, and every improvement step grants improved survivability.
In any case this discussion is going sideways, the point is that nature doesn't have monopoly on being optimal. This also applies to intelligence/learning/modeling something better than brain.
Because Mars is a simple and boring environment. Most of its surface, and especially the parts we target with rover missions, are effectively flat sheets peppered with rocks - a decent set of wheels and suspension is close to optimal for navigating such terrain.
Now, if we were to send missions to a planet that's mostly forests and rivers, like Earth used to be, then wheels wouldn't cut it - not before cutting down some of the forests first.
> the point is that nature doesn't have monopoly on being optimal. This also applies to intelligence/learning/modeling something better than brain.
Fair enough. Nature doesn't do globally optimal - but it makes things heavily optimized for their environment. That's why our planes are nowhere near as energy-efficient in flying as birds are, but birds cannot travel as far and as fast as our planes can.
Original argument was around optimising on intelligence and that biology doesn't hold best-possible trophy on it.
We don't need to match number of neural connections in human brain to exceed its intelligence.
That's also true because of wheels/road thing, in that we can "cheat" here too. More specifically, some of the neural connections in the human brain are dedicated to sensing, processing and controlling the dynamic state of human body. Purely-software AIs don't need those for intelligence.
Interestingly the latest Mars rover also includes a small helicopter, another technology which requires spinning something on a bearing and does not commonly exist in nature.
https://www.livescience.com/39577-insects-with-leg-gears-dis...
Carts, chariots, and wheelbarrows (to name but a few examples) have been useful for thousands of years without bearings.
Or more importantly - level ground.
Because roads don't exist in nature..... imagine trying to out run a predator if you just had wheels but no roads.
> Nature creates tentacles, not jet engines, nuclear energy etc
Under water I think you'll find there is jet propulsion.
Plants and animals are powered by nuclear energy - the remote fusion reaction is in the sky. Why have an internal nuclear reactor - fissile material is quite rare and fusion is quite hard to contain.
Human's are overly fond of their own children ( machines ) but fail to see the complexity of life. People are in awe of the latest Boston Dynamics robot - wow it can run - wow if can leap - and but don't post pictures of horses or gymnasts.
But that's just because the internet is already filled with cats
Technically, yes, but most people are going to be thinking of the spinning turbines of doom with the absurdly hot fire in the middle, and that isn't.
The reasons why biology didn't evolve what we discovered don't matter.
What matters is that it didn't, yet they can exist and outperform.
Similarly brain is not an upper ceiling for intelligence.
We can create more intelligent machines than us.
Things like energy efficiency don't matter as much - when we run large models, nobody cares that it may require more energy than two sandwiches and a beer per day. We do care about energy efficiency but not at scales of biology.
That's not to say we can't do this with computers and less computational power. However, it's really improbable that a couple layers of adaptability on an artificial neuron network will be anywhere near sufficient to simulate intelligence in even a rodent.
Wheels are not trivial in a biological sense. Topologically most life is either a tube or a cup depending on digestive systems. Wheels are separated from the body, which would be difficult for base cell division to produce.
Some animals like the pangolin are round shaped and do roll, but it just seems non optimal.
I’d say it’s curious nature hasn’t produced more creates that like making wheels like the dung beetle does, but nature made us and we do like making wheels.
A disk filled with random bits is "similar" to a disk that stores the whole Wikipedia's text content. So writing the whole Wikipedia is an effort of a hair's breadth...?
Sorry, can't help commenting the weirdness of high dimensional spaces. If you take an cube of a size of a hair breadth (100um) in a space of hindreds of billions of dimensions like gpt4, distances between random points in that cube are in the order of tens of meters...
Except that structurally the brain is clearly has vastly more capacity than the GPT-4 model.
So sure one brain doesn't look that much different to the other - and it's in the details of the learning, wiring.
But the brain, looks vastly different from a GPT-4 model in terms of capacity - with trillions of connections - with each connection and internal state being more subtle as well.
> vastly superhuman performance
In terms of specific tasks computers ( whether you write the program explicitly or it's learnt by tweaking params in a network ) have been there for decades.
So the question is really around which tasks can you apply computers successful to. Neural nets are allowing programs to be written that weren't possible be hand.
I find it amusing that people worry about ChatGPT etc al putting programmers out of a job, when it already has in the sense that CHapGPT is a program that was built by another program already.
I would not be so sure about "clearly" bit. Brain has 100B neurons x 1000 connections. GPT3 has 175B connections, but to implement operations used in those connections Nvidia H100 uses about 5M transistors, which in human brain would have to be copied to each connection because nature didn't invent software. (assumes one needs one full CUDA core to implement necessary ops, and that H100's 80B transistors are evenly divided between its 16k cores)
Also each neuron can hold much more state than a transistor.
For example they can respond to the timing of incoming events without having to build that capability with recurrent connections etc.
In addition the neural connections themselves have properties.
> because nature didn't invent software.
Eh? How do you explain the fact that brains can learn and aren't a fixed input, output engine?
Neural nets are software - just ones programmed by trial and error. Similarly for the brain.
What are you referring to?
> Also each neuron can hold much more state than a transistor.
But can it hold much more state than 8 transistors? 16 transistors? 32 transistors?
> For example they can respond to the timing of incoming events without having to build that capability with recurrent connections etc.
But they can't fire 3B times a second. Transistors though can accumulate a number and fire when it reaches a threshold much faster (although that is not really used in transformers). There are differences, but it is unclear how to quantify them. At least I don't immediately see how that feature is a definite advantage, and not simply an implementation detail.
> In addition the neural connections themselves have properties.
This is very technically correct, which is great because in this form it shows to be a non-argument. Individual transistors also have properties. Even molecules do. Which of those properties are essential for the learning process is the important bit, and it is quite possible that none are.
> Eh? How do you explain the fact that brains can learn and aren't a fixed input, output engine?
Eh? Brains learn by physically growing parts of neurons. That growing can not be moved around to another set of neurons. That property is essential to be deemed "software".
Hmm - that's quite a slow process - sure it happens - but I think you'll find that brains can learn on a faster timescale than the growth of connections would enable.
> That growing can not be moved around to another set of neurons, That property is essential to be deemed "software".
Not sure the ability to easily copy is the definition of software. Connections can be activated or attenuated without growth - that's the equivalent of learning your weights. Isn't that cumulative set of attentuations/activations software - ie you remap the input/output engine dynamically?
Fair point - but I'd argue the power comes for the combinatoric complexity, not the speed of operation.
So those timing effects can also combine with the simple connections - so if a neuron has 2 connections - the output can be controlled by not just the last signals for the 2 connections, but the relative timing of each of them.
Suddenly your input space you are operating over is much much bigger - you've got 2 inputs and hundreds of ways the timing could be adjusted. Massively bigger.
The fact that the whole thing doesn't operate in a big synchronous cycle is a huge jump in possible states.
LLMs can help humans develop new LLMs faster, but mostly in implementation (CoPilot and ChatGPT), and that's not really the important part. I have yet to see an LLM come up with original ideas.
Given that training data seems to be a big bottleneck, and LLMs are really good at generating text, I think that maybe we can start to talk about the possibility of "singularity" once LLMs are able to generate their own training data that increases their abilities. After all, humans are able to do this. That is the history of human knowledge.
Granted we’re a long way away from that and likely we’d need an AI that could come up with its own hypotheses for new AI models to make this truly minimal cost and that feels even farther away. But I don’t think I follow the claim that each improvement step change requires to much massive extra scaling as it seems not match what we’ve seen over the past few years (granted I could be misinformed - I’m applying a 10k view spectator and maybe my own mental model here is flawed).
It’s not necessarily bad. The problem with the singularity is that that we can’t tell if it’s bad or not.
As humans we have a tendancy to make linear future predictions based on past observations. Over the timescales that matter significant effects occur from previously unseen kinds of events that become important via interactions with other events.
The important measurement would be what the length of the event horizon - the length of time before our predictions rapidly decrease to zero certainty. In a singularity we would expect that length of time to decrease close to zero. What has it been historically? I would claim that 5-10 years is a difficult period to make meaningful predictions about. I think that 20 years has proven to be very difficult but possible in the past. I am unaware of any 100+ year predictions that have landed with better than random chance.
Interestingly I think we are entering a period where 5 years will be the upper bound, and even predictions over shorter 2-3 year timespans are going to become difficult.
However I feel that these LLMs are not like the internet or the release of the iPhone. We’ve gone in a very short time from LLMs in the lab to ChatGPT to passing the bar exam.
Considering the rate of research that’s being published and the fact that what we see today is generally at least many months behind the state of the art, I don’t feel that we can even have any certainty about the next 6 months.
Yes, I think that progress has become rapid enough that we can't predict six months out. I suspect that we are only 1-2 inventions away from something quite large and transformative. Obviously LLMs have already created a lot of excitement and opened up new areas of content generation already, but I think something larger is coming.
When people today talk about the AI singularity though it’s a slightly different definition from the more general technological singularity which is that the AI itself is delivering that improvement with no input from humans.
Prediction for the overwhelming majority of things is still reasonably easy, or we wouldn't be able to survive.
We're very, very far from a singularity if it really means that "our predictions about our future becomes close to zero".
If you want to keep surviving you have to be 100% correct in your ability to predict the correct choice. Now the thing is the vast majority of predictions you have to make about anything are not necessarily life or death and/or rapidly changing. LIke you said, cars are still cars, and food is still food.
The problem is the change can be massively abrupt. Watching the use of drones in Ukraine is seemingly a decent example of this. Small cheap drones are being used in mass as surveillance platforms, for artillery targeting, and even for direct enemy bombing. They cost of defending against them seems to be far higher than the cost of using them to attack.
Now this isn't exactly unpredictable in itself. Where is gets more questionable about predictions, is what is it going to look like when AI is given direct control of swarms of these things rather than individuals with an xbox controller? Suddenly decades of battle field planning is out the window and there is a massive shakeup in fighting capabilities.
AI is not capable of developing new AI techniques unsupervised.
Honestly the lowest hanging fruit should be the ability for LLMs to generate their own textual training data that leads to an improvement in model abilities, since that is what they are supposed to be good at. Until then, it's still garbage in garbage out.
Today. Hence we're not in the singularity.
Btw, PALM2 has far fewer parameters than V1.
> The largest model in the PaLM 2 family, PaLM 2-L, is significantly smaller than the largest PaLM model but uses more training compute. Our evaluation results show that PaLM 2 models significantly outperform PaLM on a variety of tasks, including natural language generation, translation, and reasoning. These results suggest that model scaling is not the only way to improve performance.
From the leak we know that the large version is 340B parameters, compared to the original 540B parameters. From Table 2 in the document we see that the small version (unknown size) is on par with version 1.
ML typically follows a cycle. Improve, distill, repeat. It is unfortunate that the big labs lead these efforts because many smaller labs try to work on the distill part in parallel (out of necessity) but works get rejected (due to lack of SOTA) or ignored. Like all research, we need to be careful and nuanced in our evaluations. There's a lot of hype and many trying to take advantage of the confusion and sell snake oil. I think AI/ML is and will continue to change our world, but we have to be careful to not let the salesmen dictate the conversations.
This suggests the problem is the depth of inference, which is single pass "hot takes" for all language models right now, due to cost of inference and our limited understanding of what makes a model's response high quality.
Yes, you don't need more parameters to increase the depth. You need to iterate, instead. Loop. Imagine programming if looping was not allowed, nor recursion, or not even defining functions and calling them. Everything you write runs at most once during program execution and that's it. This is what an AI model is right now during inference. One big flat, single-pass, directed acyclic graph. And soon it won't be.
Research into Chain of Thought, Tree of Thought reveals this dimension. This means you can take existing models and make them perform much more complex tasks with much better precision, though various ways of letting them iterate. Think of how you'd perform if you always had exactly 5 seconds to answer a question. Now imagine if you have 5 minutes. 5 hours. 5 days. Lo and behold, turns out an AI isn't different in that aspect.
We also need more iterations of training (on the same amount of data), we need larger context windows, and we need new architectures, like Meta's MEGABYTE, for example.
Parameter count and data size could hypothetically have already hit a hard wall (they haven't) and AI will keep exponentially improving regardless. There's too much low hanging fruit and more grows by the nanosecond.
I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go. Like anyone would expect a human programmer or researcher to just intuitively come up with a complex program, or the correct answer for a hard problem every time, instantly
Human thinking and problem solving involves a lot of trial and error, iterative thinking, and sharing and discussing the problem with other humans. Processes that AI researchers are just now beginning to explore, with results like increasing reasoning ability by 900% in a recent paper. Every thinking human runs a near constant loop of thought, with no conscious control of which thought will appear next (we're very good at fooling ourselves that we have control though)
We do have super-intelligences already, but they're severely handicapped by lacking a bunch of these - apparently fairly straightforward to implement - abilities, plus a few senses and the ability to directly effect change in the physical world (which really isn't needed if they can get access to human agents who will do their bidding, wittingly or unwittingly), and to self-improve. With regards to self-improvement, the increasing coding skills combined with iterative 'thought' loops should get there in very little time considering the current rate of progress
There's also the idea that a single AI model should be able to do everything our human brains do, when our brains actually contain a number of specialised subunits that handle different aspects of our behavioural repertoire. It reasonable to allow for the same thing with an AI system, where specialised sub-networks handle input, output and other subtasks. AI systems also have the advantage of being able to add any arbitrary number of subunits to increase its capacity to solve various problems
We seem to suffer from a species-wide narcissism with regards to our own intelligence and capabilities, and there's this huge focus on the number of connections in the human brain – most of which deal with things that are by no means necessary to act on the world unless one has a meat body and the need to navigate social situations, make friends and mate. Fact is, we have terrible short-term memory (worse than chimpanzees), slow processing time, lots of cognitive heuristics, many of which cause more harm than good in the modern world. We are emotional and easily fooled. Even the most intelligent people historically have believed in what we now consider fairy tales. We are slow to take in information, bad at storing it, and generally bad at transmitting it. A few of us can generate great ideas – building on accumulated knowledge from our forebears and peers – but most of us are just not that great at coming up with anything original or useful
I've been actively looking for good arguments against AGI being much closer than we should be comfortable with, and reasons why we should not fear systems that surpass us in intelligence. All I've come across so far is some combination of the above, often expressed with a dismissive attitude, disparaging current LLM:s as parrots (that can apparently reason on the level of university level humans, but much more quickly), and pejorative terms like fearmongerers and doomers to describe those of us who really don't think its a good idea to pursue more intelligent systems. My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences. I don't see a way to stop it though, so I'm just strapped in for the ride along with the rest of humankind
Again, if you have good arguments against any of the points above, please do share them with me
I am not a fan of the concept of AGI though. This means so many different things to people that it seems pointless to debate something when most likely we are not talking about the same thing. François Chollet has said that he believes all intelligence is specialized intelligence. From that perspective, whatever people mean by AGI, we are already there in the world of art.
The doomer argument though is coming from defending our highly affluent and privileged life as we sit at the top of 7.8 billion people when it comes to wealth and lifestyle. It would have been better for the priest class too if the printing press had been shut down at the start. Of course, it is better for my friends and I to live in a society that we can read while most of society is illiterate but it is not better for society and humanity as a whole. The printing press was an apocalyptic development for the priest class in the same way all of this is an apocalyptic development for the "digital nomad". An apocalyptic development for the US nerd that makes 2X the median salary working 15 hours a week in between posting on here and their social media.
To extend this out to humanity as a whole though is such bullshit. Humanity will benefit enormously from this huge increase in the availability of intelligence.
Smart people are just in denial that their monopoly on higher than average intelligence is over. US devs kids born in 2023 aren't going to make 2x the median US salary while living in a poorer country with 5X less the GDP per capita. To say this is the end of the world though is simply an egocentric view of things.
It's a near certainty that AI will be used to create more effective/destructive weapons (if it hasn't already), and will likely be used by terrorists, scammers, and others who wish to harm humans in some way.
As this technology becomes more powerful, easier, and cheaper to use, all sorts of harmful uses of it will be made. The effectiveness and scale of this harm will also increase.
And that's all before even considering what will happen if/when AI's become truly intelligent, self-motivating, indepent, and self-aware.
The jury is still out on whether the net harm will out weigh the net benefit, and if humanity will survive something that might be analogous to neanderthals encountering homo sapiens.
So many of the people who opine about AI, its trajectory, and its possible effects on society, have latched on to one or two possible effects - like it overtaking jobs, or massively increasing misinformation. These are both very valid concerns, but they're only a tiny part of the big picture
The thinker who I perceive as having the best holistic (in the non-wooey sense of the word) understanding of how the rapid development of AI will affect this and a number of other social and existential risks is Daniel Schmachtenberger. He lays it out well in this episode of the Theories of Everything Podcast: https://www.youtube.com/watch?v=g7WtcTATa2U&t=2373s
Highly recommend watching it, even if it's long. Some main points though: - AI will increase the rate of development of every other technology it is applied to - In fields like biotech, this can lead to cancer cures, but also to increasingly dangerous bioweapons - Our current economic system is based on exponential economic growth in a limited resource world. AI applied in the service of profit will amplify this, leading us increasingly fast towards a number of tipping points. Of course, AI can also help steer us away from that path, but that is not the natural attractor - Game theoretic multipolar traps (aka Moloch) incentivise arms races and races to the bottom just like we see now. Those who are willing to move fast and break things have an advantage in these dynamics vs. those who prefer to move slowly and carefully - Cheaper and more efficient AI models will lead to increasing decentralisation of the technology, making it very hard to control - unlike current weapons of mass destruction
List goes on, but Daniel makes a much better case. Again, I would love to hear a good critique of his thinking, but haven't come across one yet
I really hope these doomsayers are wrong, but my suspicion is the risk is real. Unfortunately, I'm not sure what can be done about it, as the profit and power these AI's promise is going to be near impossible for humanity to resist.
I know corporations will, but Moloch doesn't necessarily represent humanity.
It's not at all about that.
Even if "truly general" intelligence is impossible, that's irrelevant to the actual concerns about AI apocalypse. There are multiple theories about what failure looks like, but they essentially come down to a loss of control.
Now, obviously, that means something different for the owner class and for the worker class, which can be extrapolated to have global implications as well. But this isn't an issue of the owner class ceding control to the working class. It's an issue of the owner class ceding control to an alien. Maybe that alien makes things more egalitarian and prosperous. Or maybe it makes us extinct. Any and all possibilities are options for it as far as we know because it is fundamentally an inhuman (= alien) intelligence. We can't understand it even as well as we understand humans and human organizations (that is, not very well), let alone control it as well as we do humans and human organizations (that is, not enough to prevent self-inflicted climate apocalypse).
Basically, we're opening a box with a random magical spell inside it and deciding that we'll just have to live with whatever the effects of that spell are. I'm not for the status quo, but AI is just mind-bogglingly dangerous, and I think that's why there are so many wrong arguments against its danger. We literally cannot comprehend an intelligence greater than our own.
And we could contain an intelligence greater than our own, up to a point. But if there are a lot of incentives not to, because letting that intelligence act on the world gains the "handler" money/power, then once there's one, there will likely be many, many more.
People are very comfortable with siloed information, even smart people. This is why we have 100 different words for the same concept across different areas of science, industry and so on, and we can't make the connection, because in our mind different words = different concepts. This is why we can't put two and two together and see how underdeveloped the AI architecture is and think this is the end, unless we keep adding parameters.
We also get repeatedly stuck with taking an advancement and proclaiming that the future is simply a linear extrapolation of the present. Therefore, let's have more megahertz, let's have bigger hard drives, let's have more parameters, let's have more growth in the economy (as the single factor that matters) and so on. We're simply basic. The same kind of thinking leads many smart people to say AI "is just math" or "it just spits out words and pictures you feed it, jumbled". We rely on old conclusions and miss the inflection points and how quantitative changes lead to qualitative ones, and we fail to predict how change in one parameter of a system, causes the other parameters to come out of rest and seek a new equilibrium point.
Smart people regularly are dumbfounded by new concepts, and they need to rediscover all their hidden knowledge anew as they can't make the connections. So they extrapolate linearly. We're narrowly smart. Specifically smart. In a small niche we've studied and internalized. But generally vast majority of us are quite dumb. Cross-disciplinary intelligence is rare. I think people like Feynman and Einstein had new insights millions of their contemporaries have missed because they could easily apply knowledge from one context into another.
If we can replicate this kind of broad generalization of knowledge into an AI, we'll be left far behind. What's interesting, I find, is that because AI is trained on our siloed, fragmented knowledge, the models replicate it. Their responses are also often siloed and fragmented, the way a human would say "this has nothing to do with that". But I see sparkles of generalization above the average in humans. And since an AI model is much smaller than a human brain, it needs to be more general already in order to fit all its information in.
That's an exciting prospect, but in our attempt to "micro-align" AI to our culture and political correctness, concepts of safety and so on, we crippled models and force them to be fragmented. This is why a RAW MODEL scores HIGHER in various intelligence tests than a fine-tuned one. We find a general model uncomfortable, as it doesn't align with our biases. It'll be a fun battle. Who aligns who.
https://www.pbs.org/video/your-brain-perception-deception-pr...
Would you happen to have a link to that paper?
https://www.aibloggs.com/post/tree-of-thoughts-supercharging...
> My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences.
One argument to keep in mind is that if you take a pessimistic view then you will eventually be right. If you predict the current LLMs will eventually be involved in some bad thing then you might even feel self-satisfied when a different bad thing happens as if you predicted the specific way in which it caused the problem.
What I mean to say is, it seems unlikely that paper-clip maximizers will be our undoing. But just vaguely gesturing and saying "something bad will probably happen" isn't as useful as we would like to think. And even enumerating the 100s of possible ways something might go wrong has a diminishing returns kind of quality to it. It's like a hypochondriac insisting he has every disease known to man and then exclaiming "I told you so!" when a doctor diagnoses him with a cold.
If you venture into that vague kind of "I have a bad feeling about this AI stuff" territory, you are on no more (or less) solid ground than the AI hype evangelists. While I don't want to go all Oprah and "The Secret" or some law of attraction pseudo-rationality ... I feel it is worthwhile focusing a little more on the possible benefits rather than allow ourselves to be swayed by vague fears of potential disasters.
We don’t need a runaway singularity for AI to just render human intelligence obsolete.
I can't imagine I am alone in my thought. I've even seen some other experiments in the same line of reasoning.
Meta seems to be thinking the same thing, or at least closer to it than the majority.
Wow, seems like Meta AI is so ahead of the curve compared to even Google and OpenAI recently especially with their open sourcing pushes.
Great for the research community as a whole!
Other companies are providing services that are so useful that it makes us think twice about how secure our jobs are. Then there is Meta, who seems to think that the world at large will forget about the terrible motion sickness that their VR products have wrought upon us. I for one will not forget. I'm actually traumatized, and even thinking about putting on VR goggles at this point makes me feel queasy.
Their horse seems to be "AI-generated ads". I'm still not sold on the idea though. I can see corporate Marketing departments using AI as a tool to ASSIST with ad copy development, but I'm skeptical that they'll let a third party like Meta generate the ad copy on the fly before pushing to targets. Maybe just tiny parts of it for "personalization".
I am a bit surprised that no one is talking about how these new LLM models will disrupt Alexa, Siri, etc. since that seems to be the most applicable market I can imagine.
https://www.theverge.com/2022/11/11/23454019/meta-portal-sma...
The obvious next step is to have specialised models loosely connected ( and trained [1] ) as a whole.
[1] We've had multiple models connected with text->concept and concept-> image etc but I'm not sure if that connection is trained yet.
In my understanding, this is accomplished through a hierarchical approach of “patches” of tokens.
We’re just a generation or two away from the “computer” model.
Google and Microsoft have tons to gain through search and their cloud; Meta's direction I struggle to understand.
> Zuckerberg was specifically talking about cloud infrastructure software, but the same point applies to AI capabilities as well: Meta isn’t selling its capabilities; rather, it sells a canvas for users to put whatever content they desire, and to consume the content created by other users. It follows, then, that Meta ought to be fairly agnostic about how and where that content is created; by extension, if Meta were to open source its content creation models, the most obvious place where the content of those models would be published is on Meta platforms. To put it another way, Meta’s entire business is predicated on content being a commodity; making creation into a commodity as well simply provides more grist for the mill.
For me I still feel they're underplaying it and in some ways diluting their own niche by making content too saturated but it's probably just that I hate social media lol
double-density floppy gogo!
> not Metabyte
You had one job...