There are some implementation details about Databricks’ Photon engine in this paper: https://cs.stanford.edu/~matei/papers/2022/sigmod_photon.pdf