troiskaer

Interesting approach on passing data between steps and constructing the overall graph - it will be interesting to see what the take rate is between the two approaches (of sematic and metaflow). On the UI front, Metaflow generates viz for all objects by default in @card; but how does Sematic package up PyTorch referenced in the example (https://docs.sematic.dev/real-example) for execution on the cloud? IIRC, Metaflow packages the cwd (in addition to @conda, @pip etc.) and relies on existing packages for local execution?

Edit: Digging deeper, Sematic relies on Bazel (https://docs.sematic.dev/execution-modes#dependency-packagin...) and needs a BUILD file to specify all the dependencies for cloud execution. It seems that the entire pipeline will execute as a single (or multiple) k8s pod(s) using the same environment?

I am quite interested in trying out Sematic. Any guidelines on what kind of scale Sematic can support today (and the near future)?

How does Sematic compare to Metaflow? it optimizes for many of the same goals of Sematic - local workflows, cloud access, lineage tracking, state transfer etc?

What's your argument for Twitter/Uber/AirBnB being indisputably more technologically influential than Netflix? And let's please talk facts rather than opinions.

This is pretty much what both Netflix and Spotify do. I would argue that there isn't a canonical recommendations stack that FAANG is converging towards, and that's a direct corollary of differing business requirements and organizational structure.

> But none of them are particularly important

Nowhere was the argument made that somehow Netflix was more influential than Twitter/Uber/AirBnB, but your counter-argument that somehow it's less influential because you haven't heard of/used some projects directly holds no ground.

as well as to ML - Netflix Prize (https://en.wikipedia.org/wiki/Netflix_Prize) and Metaflow (https://github.com/Netflix/metaflow)

thanks for the unnecessary advertisement.

Seems like Kedro has a similar thesis to Metaflow - I will look into it.

How does Kedro compare to MLFlow and Metaflow?