22,000-wide output sourced by a 4096-wide embedding
You will want to use hierarchical outputs in this case. Take a look at Hinton's 'Knowledge Distillation' paper.
You will want to use hierarchical outputs in this case. Take a look at Hinton's 'Knowledge Distillation' paper.