I tried averaging the weights from a bunch of differently-trained neural networks for a Loss.jpg detector once.

If you think about what sort of brain you'd get if you "averaged" a few hundred geniuses' brains by blending them into a soup and adding some gelatin to a brain-sized sample, yeah, that's about the level of intelligence I saw in the "average neural net." It was basically in a constant state of seizure.