It was. Around the time this came out, something like half of the new ML papers were about reinforcement learning. The problem is that it’s incredibly slow and inefficient compared to any learning where you have access to a gradient and can use that to choose more targeted weight updates.

But there are certain applications where it’s the only good way of doing it (for example in games, where you don’t have access to a gradient over the space of how good a certain move is given the current game state, and it’s relatively quick and and efficient to simulate the evolution of the game).

My understanding (not an expert) is a lot of problem domains have very sparse / infrequent rewards - imagine if the only reward you gave a minecraft agent was when it mined a diamond, it would take a lot of gameplay for it to randomly do that and get a reward. So researchers spend time tuning the reward space (oh you mined some dirt, here's a tiny reward. Oh you mined rock, a greater reward, etc) but it's kind of akin to hand crafted feature detection from the pre-neural network days. The Q* mystery is did OpenAI 'solve' reward modelling the same way neural networks solved feature detection.

Well Q learning has been incredibly fruitful over the last decade. If you want to know why it wasn’t fruitful before the last decade, the answer is that it was. Data driven techniques went in and out of vogue for this kind of stuff. The difference this time is how much data we have due to the internet and smart phones and now we have computers capable of drinking from the data fire hose