Both are tricky subject and lot of cooking-like thinking (there are popular recipes, general rules, but in all practical cases to master it there is a lot of non-transferable experience and knowledge).

Some pointers:

* http://rishy.github.io/ml/2017/01/05/how-to-train-your-dnn/

* http://www.alexirpan.com/2017/06/27/hyperparam-spectral.html

At deepsense.io we are developing Neptune to facilitate that process (compare models, version-control them), here: https://go.neptune.deepsense.io (a public release in less that a month!).