Rich Sutton joins John Carmack's Keen Technologies

Now that Twitter has gone subscriber only, we can't see any context around tweets. I think they should be DOA now when posted to HN as they do not allow for follow up or rtfaing.

This is massive, Rich is probably the singular guy that legitimately has a realistic and long term vision for AGI (and has been mostly correct his whole career)

Notably he left the US because he did not want to support the Iraq war and established Alberta as THE reinforcement learning hub since then. Also he did just expand the lab so I'm curious how this will be integrated.

Carmack + Sutton working on AGI tells me it is 100% going to happen.

I also think his framing of the counterarguments is not charitable. The serious AI-risk arguments do not argue that a super-intelligent AI will necessarily be evil. They only argue that its motivations will be unaligned with ours, that it will be more competent in achieving goals than us, and that this will be bad for humans as a side effect. I think a good comparison is humans building a highway that incidentally crushes an ant colony. They didn't set out on an evil mission to destroy ants because they hate them, it just happened as a side effect of something the humans wanted. No evil required.

Off topic, but: I used to think AGI was likely, but probably not until maybe 2040. After doing a deep dive into LLMs in the last 2 years, and generally deep learning for the eight years before that, I now think there is some real chance of having something that I consider AGI by 2030. Lately LLMs have become a little useful for graph datastore, KGs, etc. I think the missing piece is the ability of LLMs to handle reasoning, and seeing GPT-4's ability to write Prolog code given a problem description, it doesn't strain my imagination too much to think a breakthrough for reasoning could be here in a year or less.