Hacker News Reader

Comment by vtuulos | Hacker News Reader

hey, I'm one of the authors of Metaflow. Happy to answer any questions! Netflix has been using Metaflow internally for about two years, so we have many war stories :)

View on HN

Thanks for open sourcing this.

Can you say a little about which niche this would occupy, and what the motivation is? Is it intended to compete with Tensorflow and Pytorch or to be an industrial strength version of SKlearn.

I looked through the tutorial on my mobile and the answer was not immediately clear.

Is the benefit that it auto scales on AWS without having to think through the infrastructure?

Hi omarhaneef, We don't intend to compete with Tensorflow, PyTorch, SKLearn. What we offer is a way to iterate and productionize your models written using any of the aforementioned libraries (and more). https://docs.metaflow.org/introduction/what-is-metaflow contains further elaboration of our philosophy. Auto-scaling infrastructure is one piece of the puzzle, and Metaflow goes beyond that offering a comprehensive solution for model management.

G'day, seems like an cool tool, thanks - the links to the github tuts are currently broken...

I'm of the opinion that adopting some standard DAG meta format for data science may make a positive impact on the reproduceability issues we have in science generally. So its good to see the idea has real world merit as well.

Hi ageofwant, Thanks for taking a look at Metaflow. We will fix these links. Can you point us to the offending links?

I have updated the links.

Pray you do not update them further?

Hey I meant to track one of y'all down at the MLOps conference, but didn't get the chance. I've built a very shitty version of a cached-execution DAG thing internally, and one of the design decisions I made was to have it so that parent nodes in the DAG don't need to know anything about child nodes. This allows for larger DAG builders to be more easily subclassed.

MetaFlow doesn't do that -- instead each 'step' has to know what to call next, which means that if I wanted to subclass e.g. the MovieStatsFlow in [here](https://github.com/Netflix/metaflow/blob/master/metaflow/tut...) and say, add some sort of input pre-processing before the compute_statistics call, I'd essentially end up having to either override what compute_statistics does to not match its name _or_ copy-past e that first step just to replace that last line.

I'm sure this design decision was considered and/or that use-case doesn't come up a lot at Netflix (although I've encountered that a lot), or maybe I'm missing something very obvious, but I'd love to hear your thoughts on that.

We erred on the side of simplicity to keep things manageable for our users.

That seems more complicated.

I have been looking for something exactly like this, but I use GCP not AWS. Is there a way to deploy outside of AWS? What would be involved in getting it to work with a different cloud?

All of our integrations are driven by plugins. We are exploring our roadmap with regards to integrations with other clouds.

Hi Ville, I cannot express how happy I am that you open-sourced Metaflow. I have three questions: 1) Do you know ETA of R API release? 2) Would you recommend using both Metaflow nad MLFlow in projects? Could you please explain why yes or why no :) 3) Do you plan to release integration with Spark/Yarn?

Thanks in advance, Roko

Does this do experiment tracking similar to MLflow? I’m trying to figure where these two overlap and where they diverge.

yep, Metaflow tracks everything: Your code, dependencies, and the internal state of the workflow automatically.

A big difference between Metaflow and other workflow frameworks for ML is that Metaflow doesn't only execute your DAG, it helps you to design and implement the code that runs inside the DAG. Many frameworks leave these details to the data scientist to decide.

The centralized DAG scheduler seems like a pretty important part. How much will not having Meson hamper the usability?

Hi bhtucker, We do have plans to integrate with production-grade schedulers in the very near future. https://github.com/Netflix/metaflow/issues/2

note that Metaflow comes with a built-in DAG scheduler, so it is perfectly usable without a centralized scheduler like Meson.

Integrations to Airflow or StepFunctions are on the roadmap, depending on what seems to resonate with people outside Netflix.

Thank you for sharing this. Would this be useful for me if I only need the deployment management part? Don't really need to track experiments, just looking for an easy way to deploy my models to Fargate.

Yes indeed. You can access your model via the Metaflow client inside your service - https://docs.metaflow.org/metaflow/client. We are looking into releasing the hosting component on Metaflow in the future - https://github.com/Netflix/metaflow/issues/3

Reply on news.ycombinator.com