Neither does GPT-4 or other sparse mixtures of experts, such as e.g. switch transformers [1].

[1] https://arxiv.org/abs/2101.03961

Speak for yourself :)