lucidrains and Ice Cream are my references in terms of research, knowledge and productivity. Phil was always available to guide and hear me. One time I told him about an underground research in another language and he was kind enough to check if it had any merit.

About X-Transformers, it is a very great piece of engineering that implemented almost of all possible improvements in transformers. But according to my experience and Phil himself, only the Feedforward GLU and RoPe (Rotary Positional Embeddings) works (or to be fair, they show improvements in more general use-cases)