Ask HN: What's the state of online RL with no pretraining

What's the current state of online continuous reinforcement learning? And where can I read more about it. I've seen a lot of research papers within the last few years on training RL models with offline data, or at most, episodic training. From what I've gathered, It's just too difficult to train agents online in complex environments because the policy conditions the training data. I think we're kicking an important can down the road for short term gains by not tackling continuous online RL.