Ask HN: How do you do hyperparameter searches?

I know how to do hyperparameter searches.

This is a survey. I'm curious how other people do hyperparameter searches in real life.

1. What algorithm do you use? Do you do it by hand? Do you use random search, grid search, Bayes search or some iterative process? 2. Do you cache anything? If so, what's your process?