One thing that's changed is:

> So now we have the top machine learning research institutes, DeepMind and OpenAI, still spending the majority of their time and resources on Deep RL

DeepMind has diversified at least some since 2019, and I'm fairly confident that OpenAI is spending more resources on huge transformer models than on RL these days.

Which is really the only thing that has changed, since even in 2019 there are at least a dozen world-class institutions doing AI/ML research aimed at addressing issues raised in this blog post (and others).

The blog post is accurate about OpenAI/Deepmind c. 2019, but is wrong about the overall composition of research effort in the field c. 2019. Outside of two small and very new labs, most ML research wasn't focused on RL, and most RL research wasn't focused on DRL as a silver bullet.

Sort of of the west coast SV version of only paying attention to work out of MIT and Stanford and therefore missing most of the interesting things happening in the world.

I think when looking at least at large scale applications in the context of games (and not just super-expensive showcases), like in the Stockfish chess engine, we see that it's not primarily about depth, it's about architecture design. Reference: start here https://stockfishchess.org/blog/2021/stockfish-14/ and go down the rabbit hole...

In the broadly useful domain of recommender systems (which typically make use of some type of RL-like feedback loop, but can be implemented using simple clustering approaches), at least in 2019, neural network-based approaches didn't seem to fair too well, either: https://arxiv.org/pdf/1907.06902.pdf (arXiv pre-print, but this is an award-winning paper).

Since then, it seems that researchers are moving away from getting deeper and deeper (the low-hanging fruit), and try to be more creative instead: new architectures, combining symbolic (logic-based) and sub-symbolic (ML-based) AI, etc.

Most of the improvements in real world have been by improving data representation for perception (improvements in transformers, self supervised learning), so so far the article seems right.

The article is about the failure of reinforcement learning to make it out of games and into any kind of real world task, so I think alpha go doesn't really change the argument.