The SQL with pipe syntax is also been implemented at Databricks since Jan 30, 2025: https://docs.databricks.com/en/sql/language-manual/sql-ref-s...

Still, the best is yet to come. Previously, SQL extensions were a pain. There was no good place, and table-value functions were a mess.

Now, it would be possible to have higher-order functions such as enrichment, predictions, grouping or other data contracts. Example:

  FROM orders
  |> WHERE order_date >= '2024-01-01'
  |> AGGREGATE SUM(order_amount) AS total_spent GROUP BY customer_id
  |> WHERE total_spent > 1000
  |> INNER JOIN customers USING(customer_id)
  |> CALL ENRICH.APOLLO(EMAIL > customers.email)
  |> AGGREGATE COUNT(*) high_value_customer GROUP BY company.country

This may be called one SQL to determine distinct e-mail domains, then prepare an enriching dataset and later execute the final SQL with JOIN.

Iterative SQL with pipes may also work better with GenAI.