In Weaknesses of RNN architecture I’ll motivate the need for the transformer architecture. In Transformer architecture I’ll go more in depth into its architecture. Finally, in Experiments and results I’ll show how the transformer performance compares to the RNN performance, and as a sub-research question, how the original encoder-decoder transformer architecture compares to GPT-style decoder-only architecture in Encoder-decoder vs decoder-only