my experience is that while data keeps growing at an exponential rate, its information content does not. In finance at least, you can easily get 100 million data points per series per day if you want everything, and you might be dealing with thousands of series. That sample rate, and the number of series, is usually 99.99% redundant, because the eigenvalues drop off almost to zero very quickly after about 10 dimensions, and often far fewer. There's very little reason to store petabytes of ticks that you will never query. It's much more reasonable in many cases to do brutal (and yes, lossy) dimensionality reduction _at ingests time_, store the first few principal components + outliers, and monitor eigenvalue stability (in case some new, previously negligable, factor, starts increasing in importance). It results in a much smaller dataset that is tractable and in many cases revelatory, because it's actually usable.