They use DeltaLake + Spark 3.0, and are mostly careful to partition well.

Their datasets are small. Most tables are ~50GB, the odd table up to ~2TB. The clusters typically are nothing shabby for this size, defaults to ~[4-12]x32GB.

The queries that I have seen are typically not written well. Think view-on-view-on-view (there's a BigCo policy against them materialising data..), and where the filter is applied in the last step. The stuff of horrors, but something I've seen in more-than-one-BigCo.

But we have compared some of those same queries on BigQuery vs. Databricks, and, I don't know if BigQuery's execution optimiser is better? Or if the BigQuery storage is better organising the data? Or if BigQuery is simply throwing more resource their way?