The Delta Lake is a marketing term from Databricks to, in part, market their Delta file format and all the clusters you’ll be spinning up.

Delta files are actually an amazing format though. At their most basic, they took Parquet files (columnar database) and let you stream off them really easily. Which takes a lot of complexity out of your pipelines - dont need Kafka for everything, don’t need to figure out when new rows get added (or a whole other set of jobs around that).

But using Delta files really can change the way you develop pipelines (and ML pipelines), so I forgive them for inventing a new term.