I would say that BERT was an incremental improvement (as well as many others). But the idea of much more efficient sequence processing was introduced in this exact paper.