Glad to see this articulated so well. The 'Overall' paragraph sums up thoughts that had been in the back of my mind for months. Plus, hey, it's Gwern. If you're reading this: you're an inspiration, man.

What is still unclear -- to me at least -- is the technical challenges that lie ahead of this "neural networks all the way down" approach. I get the impression we'll need quite a few breakthroughs before usable Agent AIs are a thing. Insights on the order of importance as, say, backpropagation and using GPUs.