That's what a "mixture of experts" has always implied to me - a vast amount of overfitting.