Hacker News Reader

>It's a shame how data hungry DRL is (even when compared to DL), but the DRL framework encompasses all the standard classification/regression tasks and also includes decision making, planning and pretty much anything else you can think of.

Sure, and a Turing machine can express any program you care to write. RL seems like the Turing tar pit of machine learning: theoretically able to express everything, practically convenient mostly just for trivial examples.

>Model generality and data efficiency are in an inverse relationship and a lot of research has been in moving up or down this hyperbola.

Sure, basic Curse of Dimensionality, but the whole success of hierarchical modeling in the brain, hierarchical Bayesian methods, and deep neural networks has been that hierarchical modeling seems to ameliorate or even defeat the Curse of Dimensionality. The question is: well, why can't it do that in reinforcement learning?

In interesting terms: why does the teaching signal have more information in supervised learning than reinforcement learning, relative to the inherent uncertainty of the task?

DRL is too fragile to be realistically useful while supervised learning is not.

but the DRL framework encompasses all the standard classification/regression tasks and also includes decision making

Except labelling data, correct ? which is non trivial effort.

> Except labelling data, correct ? which is non trivial effort.

Choosing choice of labeling is a RL problem too.

If you're choosing environment actions to learn how the environment 'labels' them, that's the classic topic of 'how do we make DRL models explore' well (and arguably is the Achilles heel of the model-free DRL approaches OP is criticizing: the NNs can easily learn to optimize their actions, even tiny NNs are more than enough, but they just don't get fed the 'right' data ie. exploration is bad). Relevant papers: https://www.reddit.com/r/reinforcementlearning/search?q=flai...

If you're being very narrow and considering a classification problem, well, that's a RL problem too: you can optimize which datapoints you get labels for based on how informative a datapoint is (most datapoints are simply redundant) or how expensive it is to label. That's called 'active learning': https://www.reddit.com/r/reinforcementlearning/search?q=flai... It's particularly natural if you are doing large-scale image classification and have a service like Amazon Turk plugged in to get (or correct) labels.

In many cases people greatly overestimate the effort of labelling data. For the price of just a single engineer man-month you can generally label a lot of data, sufficient for many tasks; especially if you can do bootstrapping (the humans don't label all data, but correct mistakes of a previous insufficiently good automated solution) or some transfer learning from a somewhat similar task or an unsupervised one.

Like, if you need a solution for a some niche of image classification, even a single afternoon of labeling data might be sufficient to adapt an ImageNet classifier to your particular labels and get reasonable accuracy.