Hacker News Reader

Comment by sillysaurusx | Hacker News Reader

Tempted, perhaps, but not enough to join. Carmack's working on AGI, and he's committed to a decade-long time horizon.

The impact of AGI is far greater than rockets, too.

Elon is working on AI too, and Dojo is one of the most exciting things in the field if you ask me. Carmack has always been good at leveraging hardware to do amazing things on the graphics side, and AI hardware is not so dissimilar to graphics hardware.

Elon is working on ML, not AGI.

Carmack is working on artificial general intelligence. As far as I know, that's well outside of anything Elon is doing.

I mean, Elon just announced that they're building a literal android that is supposed to take verbal directions and perform everyday tasks for you. If that's not AGI, I don't know what is. It may not be very likely to happen anytime soon, but the ambition is there at least.

If you haven't watched the Tesla AI presentation, I highly recommend it. The dancer in a spandex robot suit is getting all the headlines but the technical presentations are the real deal. Karpathy in particular is an excellent technical presenter and he didn't hold back.

> Elon just announced that they're building a literal android that is supposed to take verbal directions and perform everyday tasks for you. If that's not AGI, I don't know what is.

It's not AGI. An AGI is something that can think for itself, the way you can.

Elon described this robot as taking directions like "go to the store and get me this list of groceries", implying that it would be able to, like, read the list, choose a store, drive a car there, use a cart, locate the groceries, go through checkout, etc. It's going to need to do a lot of thinking for itself to complete tasks like that. And more generally he described replacing human labor to the point that a universal basic income becomes necessary. AGI is definitely the ambition here.

Nahhh... I'll paraphrase my old skool A(G)I advisor from my first year in grad school: you tell ML to get your groceries; AGI tells you to get the fucking groceries.

Its not AGI. It's machine learning. It's unknown if it's possible to ever achieve AGI by using machine learning techniques.

Seriously, downvotes for a 100% factual statement?

I think what's more likely is we'd discover how many common, useful tasks you could perform with a humanoid robot that don't require AGI, with a more Alexa-like approach of iteratively adding support for explicit discrete tasks over time. We've already moved back the goalposts for AGI in so many other areas as we realise how many tasks can be decomposed to something quite mechanical in the right vector space etc.

If anything, the lesson of the entire history of robotics is the exact opposite. I'm surprised to see so many people believe that a robot doesn't need AGI to replace most human labor. Or maybe people just don't believe that Tesla's general purpose robot is a real project. Which is a view I'm partially sympathetic to.

Elon's been giving some lip service to closer-to-GI capabilities anyways, e.g. when describing how it would be optimal to be able to control Tesla's humanoid robot by saying "please go to the store and get me the following groceries"...

https://youtu.be/j0z4FweCy4M?t=7775

Whether Tesla wants to fund serious work towards that may be a different question (but they do seem to be serious about at least building the robot).

> and Dojo is one of the most exciting things in the field if you ask me

Why do you think this?

Not the person you're asking, but

- Long iteration time kills productivity, a lot of AI work has long iteration time because hardware is slow.

- It's possible that bigger models on faster hardware are actually all we need to solve some problems. For example see GPT-3.

I believe that the major limiting factor in the speed of development of AI is hardware, not software. Now and for the foreseeable future. The most important projects in AI, therefore, are the ones pushing hardware performance as high as it can go, and the people who have the opportunity to write software for that hardware are going to make the fastest progress.

Though Carmack probably doesn't agree with me on this given his statements that "I believe that current supercomputers are actually sufficient for human level AGI" https://twitter.com/ID_AA_Carmack/status/1340369768138862592 and "I am provisionally not very interested in custom AI accelerators, because I don't think the extra performance will outweigh the reduced flexibility and immature software compared to GPUs" https://twitter.com/ID_AA_Carmack/status/1412897975370719235

I'm tempted to agree with Carmack here, having worked at a now-defunct self driving project previously that used multiple server-grade GPUs and many-core CPUs pulling 1000+W in their car that couldn't even figure out how to move around a car that was parallel parked too far out in the road.

More teraflops are not what is needed.

I mean, I partially agree with his opinions from those tweets, but my conclusions are still different. Firstly, the AGI of the future may well be able to be distilled to a form that would fit on a top supercomputer of today. But developing an AGI is going to need a whole lot more power than running an optimized version after the fact. And secondly, the flexibility and availability of GPUs is hard to beat, but I think that after a few iterations of datacenter-scale hardware we'll learn what's needed to really make it shine. So maybe the first iteration of Dojo won't be much better than a GPU cluster in practice, but the third or fourth could be spectacular.

Sure, but this has nothing to do with Dojo specifically (over say TPUs or other matrix accelerators).

Also, I think most people who have so serious deep learning believe that larger models/faster iteration will only get us so far.

Sure, TPUs and other accelerators are also exciting. Dojo is specifically interesting because of the massive off-chip communication bandwidth it has, which I believe is important. But in general, I'm excited about custom AI silicon.

OpenAI seems to believe that scaling transformers can solve many if not most problems, so I'm not alone here.

Then don't require an AI to run in realtime. Just let it interpret the world in 0.001x speed. This way you could easily simulate hardware which will only be available in decades.

Can you explain what you find exiting about the Dojo?

Making custom AI chips (other than as practice) seems a giant waste of money at the scale of it. In addition I was kinda underwhelmed how little benefit they got compared to an off the shelf solution.

Edit: Someone else asked the same thing before.

AGI on classical computers? is this a joke?

Reply on news.ycombinator.com