I wonder if there is a principle here we can apply to neural nets like dividing up the layers for different tasks (no not MoE but something more fundamental).