Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
He's openly talked about his work ethic in a bunch of places. He's the type of guy who after a life time of coding calculated he's 100% efficient up until 13 hour work days and then he drops off[0]. Although he did mention working those long hours is often best working on multiple things instead of 1 topic but maybe with AGI there's a bunch of different avenues to explore.
You are bound by the amount of data and computational resources you have at your disposal. Neither are tied to man hours. You can stay up all night for days waiting for your model to train, and it will do you no good.
This is exactly the experience of most teams I've spoken with, be they students or businesses, for all the pre-production phase. You simply can't and shouldn't spend on costly AI infrastructure before you've nailed your solution; in fact any kind of infra not just AI.
What you do is rent some cloud to power quickly through your tests — better have 10x worth of big Nvidia GPUs over 2 weeks than buy 1 or 2 max yourself and wait 5-10x more time — not even factoring that setting up clusters of GPU and running such workflows consistently over days, weeks requires pretty deep sysadmin/hardware knowledge and experience; it took me two years to really master that non-problem part on my home server (but now it's a skill I have so that was worth it, but certainly set my research and learning back by as much time).
Besides, there's a time when the familiarity, safety and general comfort of home simply can't be beat. Notwithstanding pool tables and free soda, lol.
Unfortunately that could never happen because of the SEC.
I wouldn't be surprised if the opposite was true, at least with the theory part. AI didn't really go anywhere for decades, because people focused too much on theory.
Otherwise, there's a lot of data and hardware at your disposal, even from the comfort of your home.
> Wouldn't it make more sense for him to join a cutting-edge team, like DeepMind or OpenAI?
You mean they guys that are training with videogames that people like John developed?
Everything is easy in hindsight.
And that's without mentioning that the specific result you are talking about is an infinitesimal progress when compared to "AGI".
> It's the kind of problem that requires incredible amounts of data, hardware, and theory to make any progress.
So I point out a recent example of progress (of which the sequence of fundamental insights is nearly always incremental), where the theoreetical insight was a theoretical derivation, which could be and probably was derived on paper / blackboard, as a direct counterexample: it does not require "incredible amounts of [...] theory".
Why would you compare this with performing manual backpropagation?
>Everything is easy in hindsight
Every breakthrough is non-trivial, else it would not have been a breakthrough, and yet the breakthrough itself can be a relatively simple calculation...
The concept of "AGI" is undefined and virtually worthless to me. There is only non-trivial insights, i.e. theorem and proof.
Substitute math for anything you want.