Hacker News Reader

First off, thanks for your efforts!

From what I've followed of machine learning, I feel like there are areas that are not very rigorous, for example where humans have to tune weights and parameters. Does your work address those places?

Specifically do you think you can train on enough parameters, and compare enough results, to begin to estimate how that tuning can be done automatically?

View on HN

Yes, gradient-based hyperparameter optimization is one option for taking the human out of the loop to some extent. Of course, we can't avoid having to specify hyper-hyper-parameters!

More generally, automatic hyperparameter tuning (without gradients) is already a standard part of many researchers' pipelines.

It's also part of good scientific practice, since it makes it harder to bias the results towards a particular method.

Reply on news.ycombinator.com