I don't think most people realize this but the "old" stuff often works better, has less churn, and has far lower overhead costs for deployment than the "new" stuff. Depends on the domain and the goal.
To your point, I replaced an LSTM that required ~$100k of infrastructure with XGBoost that required no more infrastructure (we created and used the model at query time on existing infrastructure we already had for query loads) and only lost about 2% accuracy (LSTM: 98%, XGBoost: 96%). This was two years ago and it's still in use.
It's also a LOT more explainable and interpretable.