Scaling Transformers at Cohere: What I Learned

I had the opportunity to help Cohere work on scaling transformers in the last year. If you have any questions about the article leave them below and I’ll do my best to answer openly. :)