So much this. It's why data engineering ends up being more and more useful, even if it's not sexy. Automating the boring parts of dumping all the ugly data and cleaning up the stuff that is sampled is a better use of resources.