2022 Letter
zhengdongwang.com
zhengdongwang.com
To my surprise, I won the tournament, despite having done the bare implementation with no tweaks! I didn’t get a perfect grade on the assignment though - apparently I had miss implemented the queue for the alpha-beta search, and had completely reversed the queue ordering.
Of course, searching “in reverse” through the alpha-beta tree turned out to be the difference that won me the tournament. It was rather humiliating, but a good lesson on sheer luck’s role in academic approaches to AI.
Gary Marcus is an academic grifter and to me he is no different than crypto bros who grift non-experts.
Marcus has been writing some variant of exactly the same article multiple times a year for the last 15 years.
In theory, as long as you can translate your inputs and outputs to an array of floats, a neural network can compute anything. The required number of neurons might not fit into the world's best RAM, and the required number of weights and biases for those neurons might not be quickly calculated by a CPU/GPU however.
The brain itself is infinitely more complex than artificial neural networks. Maybe we don’t need all of what nature does to get there, but we are so many orders of magnitude off its redonk. People talk about number of neurons of the brain as if there’s a 1:1 mapping with an ANN. Real neurons have chemical, physical properties, along with other things probably not yet discovered going on.
I also think it is a bit strange to use the human brain as an analogy because biological neurons supposedly are booleans and act in groups to achieve float level behavior. For example I can have neurologic pain in my fingers that isn't on off, but rather, has differences in magnitude.
I think we should move away from the biology comparisons and just seek to understand if "more neurons = more better" is true, and if it is, how do we shove more into RAM and handle the exploding compute complexity.
Insights into AI algorithms drawn from hippocampal function:
Biological networks that result in conscious “minds” have a ton of loops and are constantly learning. You can essentially cut yourself off from the outside world in something like a sensory deprivation bath and your mind will continue to operate, talking to itself.
No current popular and successful AI/ML approach can do anything like this.
What is an AGI then?
A Roomba is not AGI because it can do what a cleaner does.
“Artificial general intelligence (AGI) is the ability of an intelligent agent to understand or learn any intellectual task that a human being can.”
I am more concerned though if the definition includes things like philosophy and emotion. These things can be quantified, like for example with AI that plays poker and can calculate the aggressiveness (range of potential hands) of the humans at the table rather than just the pure isolated strength of their hand. But it seems like a very hard thing to generally quantify, and as a result a hard thing to measure and program for.
It sounds like different people will just have different definitions of AGI, which is different from "can this thing do the task i need it to do (for profit, for fun, etc)"
Chat GPT allows for conversation that is pretty remarquable today. It hasn't learned the way us humans have - so what?
I think a few more iterations may lead to something very, very useful to us humans. Most humans may just as well say Chat GPT version X is Artificial, and Generelly Intelligent.
So that's just one small example that we don't need AGI to be a model of the real human brain, with synapses and blood-brain barriers and everything. Rather, we just need one system to do n number of tasks at roughly the same level as a human for it to be "general". Maybe it's not AGI, but it also is not a hardcoded robotic arm that can only work with square objects of a certain dimension.
If you had a robot that was pretrained in a virtual world, assembled in the real world, and then it begins testing and observing and resolving its own physical capabilities (moving arms and legs to stand and jump and backflip)... and then it also had a vision system to scan for threats and objectives... and then it also could resolve text and voice prompts to learn its next objective ("go get my favorite beer can from the fridge")... and the robot knows to ask you more questions to learn what your favorite beer is and also it knows how to preserve its own life in case the dog attacks it or the fridge topples over on it... then I think you have an extremely useful tool that will change the world, regardless of if it is labeled as AGI or not.
Shane Legg of DeepMind wrote a blog post at the opening of the 2010s where he stuck his neck out to predict AGI with a time distribution peaking around 2030. He thought the major development would be in reinforcement learning, rather than the self-supervised GPT stuff.
This isn't to say that there wouldn't be some simple hack to allow memory formation in chat agents, just that there's at least one advance we need besides simple scale.
[1] As in not subliminal, not anything to do with philosophical notions of qualia.
Most notably, neural networks alone will not reach any kind of AGI.
Start adding the capacity to read from massive knowledge stores, and a place to keep long term information (i.e., memory, probably also in a database), plus a feedback loop for the model to learn and improve? Plus the ability to call APIs? Now you're talking. I think all of those pieces are close to doable right now, maybe with a latency of 5s. If one of the big players puts that in place in a way that is well measured and they can iterate on, I think we'll start to see some really incredible advances.
Similarly, transfer learning is finally good.
And the models are generalist, few shot learners.
As a consequence, individuals with minimal expertise can set up a world class system to solve niche problems. That's really exciting and it's going to get easier.
I think we're at a point where there's very little excuse not to launch in many languages at once.
"At its core, GPT Index is about:
1. loading in external data (@NotionHQ, @Slack, .txt, etc.) 2. Building indices over that data 3. Inputting a prompt -> getting an output!"
For answering "queries", it appears like it iterates over the documents in the store, i.e., NOT using it like an index, and feeding each document as part of the context into the LLM.
Just as robots have had to graduate to the world of continuous SLAM, navigating while building and constantly updating a map, I feel like there's a big missing piece in current AI for a system that can simultaneously act and learn, that can reflect on gaps in its own knowledge, and express curiosity in order to facilitate learning— that can ask a question out of a desire to know rather than as a party trick.
It's not as fast a learner (efficient with samples) as humans are; it doesn't continuously learn from interactions with others like we do; and it's certainly not superhuman at everything (or probably anything other than breadth of knowledge)…
…but Yudkowsky recently criticised Musk for taking that too far and limiting the definition of "AGI" to what is normally meant by "ASI" (Yudkowsky was also saying that no, GPT isn't AGI): https://twitter.com/ESYudkowsky/status/1600362288149856256?c...
~~~~
Sure! Here are the solutions to the problems:
1. Solve for x: 3x + 4 = 10
Solution: x = 2
2. Simplify: (5 + 3) * 2
Solution: 16
3. Convert: 3/4 cups to tablespoons
Solution: 6 tablespoons
4. Round: 8.6 to the nearest whole number
Solution: 9
5. Find the area of a rectangle with length 5 and width 3
Solution: 15
6. Find the perimeter of a triangle with sides of length 4, 5, and 6
Solution: 15
7. Solve for y: x + y = 8, x = 3
Solution: y = 5
8. Simplify: (6 - 2)^2
Solution: 16
9. Find the volume of a cube with side length 5
Solution: 125
10. Find the circumference of a circle with radius 3
Solution: 18.85
11. Simplify: 1/2 + 1/3
Solution: 5/6
12. Find the slope of the line with points (2, 3) and (4, 5)
Solution: 1
13. Find the midpoint of the line segment with endpoints (6, 8) and (10, 12)
Solution: (8, 10)
14. Solve for x: 5x - 3 = 2x + 7
Solution: x = 4
15. Find the roots of the equation x^2 + 3x + 2 = 0
Solution: x = -1 and -2
16. Find the median of the set {1, 3, 5, 7, 9}
Solution: 5
17. Find the mean of the set {2, 4, 6, 8, 10}
Solution: 6
18. Solve for x: log_2(x) = 3
Solution: x = 8
19. Solve for x: sin(x) = 0.5
Solution: x = 30 degrees or x = 330 degrees
20. Find the inverse of the function f(x) = 2x + 3
Solution: f^(-1)(x) = (x - 3)/2
~~~~
Mistakes: 3, 5, 14, 19 (half points)
Answer to 9 would probably be accepted in a school exam, even though it's rounded to 2 decimals and the question didn't specifically ask for that.
So, depending on how severely you mark it, 75%-82.5%.
Can submarines swim? And does it really matter?
Also, when I was at school, 75%-82.5% was a good score, and looking up the current system, that percentage range is in the top three grades (out of 9) at that level, and can be top grade depending on year and exam board.
That is why I say you are moving the goalposts.
[0] UK secondary schools are different from US highschools: https://en.wikipedia.org/wiki/General_Certificate_of_Seconda...
Getting 3/4th of simple elementary school questions right indicates that someone is really bad at math. These are not difficult questions. I don't have time to come up with 20 exercises, but I can suggest a few that I think the AI may have trouble with. All of these are trivial single-variable linear equations that a smart elementary school student should have no problem with. (Note that the numbers are intentionally scrambled; this does not increase the difficulty of the problem):
1. Solve for x: 113(6x - 45) = 92x
2. Multiply 13.4a - 18b by 18b + 13.4a
3. Four girls bought bus tickets. Anne bought seventeen 20-minute tickets and paid $323. Marianne bought twenty-eight 75-minute tickets and paid $784. Alice bought nineteen 20-minute tickets and eight 75-minute tickets, and Alex bought seven thousand eighty five 20-minute tickets and ninety six 75-minute tickets. How much did Alex pay?
4. An old man walked fifty nine kilometres in four hours through a flat field. He drove a car to France for nine hours, and then proceeded to walk for eighteen hours through a flat field at the same pace. How many kilometres did the old man walk in France?
5. Alice bought seventy-nine pens and twenty-six notebooks. The arithmetic mean of the cost of these articles was $9.31. The sum of the cost of the notebooks was $112.762. How much did a pen cost?
Algebra and trig wasn't until secondary school for me ("year 7" we called it, school year beginning age 11), though I personally had a head start from having learned to read with the Commodore 64 user manual.
Well, if you're not willing (or well calibrated enough), I should get some old exam papers, see how well it grades against students. I wonder if there even are any downloadable pre-GCSE exams…
Edit: also, where might I find some old Polish example exams and marking schemes? ChatGPT is inherently multilingual, so I might as well.
This thread is no longer on my first page of comments, so I may forget to reply, but I've downloaded those and do intend to test it against those exams, and will put the write up here: https://github.com/BenWheatley/Studies-of-AI
A more general form of your question is whether we can get to AGI with just incremental steps from where we are today, rather than step-change way-out-of-left-field kinds of ideas. People are split on that. Personally, I think that incremental changes from today's methods are sufficient with better hardware and data, but Big New Ideas could certainly speed up progress.
But watch for lots of breathless "look how close we are!" messages.
The other thing GPT is good for is polluting our info-sphere with stuff that sounds confident, but may be anywhere from dead right to slightly off to completely wrong. Having automated means of producing high volumes of fine-sounding nonsense is not the path to anywhere good.
[above is based on chatgpt's summary of my long draft here: https://hastebin.com/raw/muzuvodupu - I also added the last line above and the parentheticals.]
I have been testing some things with text-davinci-003 which is not the same as ChatGPT but quite similar. https://aidev.codes/u/runvnc/flappy/edit
I was thinking along the same lines. Can you put some form of contact on your profile or contact me at mine, I'd love to chat with you about what you're doing.
One thing to note: I don't think a visual game is the best scenario for this because it is very bad at recognizing images, which means it is like asking a blind person to judge a painting. Not a fair task, since it doesn't have a visual cortex at all.
For example, I tried to get chatgpt to output vector graphics in svg format, but it didn't do a great job. I also asked it to recognize what object is in an svg vector image and it took a guess that wasn't bad but was far from being a suitable object detection, it didn't really recognize or describe the object in the svg graphic accurately.
Here is the svg I asked it to describe, it is the default image when you open this svg editor:
I copied this default image into chatgpt and asked it to describe it, and likewise you can ask it to generate its own svg, same as code, but it does very badly.
For the example image, it thought it was a mobile phone, whereas it was a depiction of a piece of paper with the corner bent down and some writing on it - a mobile phone is not a bad guess but far worse than it does for language-based tasks, and it didn't describe any part of the square, the circle, or the text "SVG".) I just repeated the experiment and got this interesting transcript you could look at if you want: https://hastebin.com/raw/cuqodivotu
As you can see it just does okay, not great.
So instead of these types of tasks, I think the supervision should be something that does not involve any imagery at all but is still a sort of generic puzzle. If you have some form of contact I'd love to chat with you about this area of research (this is not job-related).
If it's possible maybe you could click the Discord link in the upper right hand corner of that web site? That is my new Discord. I am runvnc in there. If not I am in the OpenAI Discord under the api-projects. You can also just email me runvnc at gmail dot com.
Yeah, it's actually kind of funny when you ask it to make something with SVG. It can get some of the shapes in there but doesn't know how to arrange them at all. I was aware of the lack of visual data in the training etc, flappy was just kind of a late night random idea.
But I have done some simple tests with for example telling it to write a a few basic unit tests and putting it in a loop. Also was kind of a mixed result but it depends on how you do it.
Would love to discuss this stuff whenever.
It needs many inputs and many outputs into the real world. A true AGI sees the reaction of the world to its own actions. That is the root of sentience.
Are there any notable figures in the space that didn’t go or get into top schools? Feels like it’s becoming an elite echo chamber.
https://www.ibm.com/common/ssi/cgi-bin/ssialias?appname=skmw...