Speaking of bagging: Geoff Hinton talks a bit about model averaging and has a very stimulating pair of hypotheses related to this, which are:

a) genetic recombination in the form of sex might be "bagging across genes" that prevents tight co-adaptation (= overfitting on the evolutionary timescale)

b) the brain uses noisy discrete firing instead of continuous communication because it allows for "bagging across network topologies".

The bulk of his talk is about how dropout in neural nets can be used to accomplish some of the same performance benefits that model averaging gives on other algorithms (at a much smaller relative computational cost).

Here's the talk: https://www.youtube.com/watch?v=DleXA5ADG78