Hacker News Reader

Mixing these would be pretty cool. Further reduced compute for MoE while keeping the performance.

In the paper they already show a mixing of these two with Mixture-of-Depths-and-Experts (MoDE).