Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.