Hacker News Reader

Comment by ericcolson | Hacker News Reader

love the transparency this brings. Any 3rd party tools with plans to integrate with it? (e.g. analytics layer companies?)

View on HN

Thanks! Lots of plans in the works. Two directions we're moving:

1. Integrate with orchestration systems (E.G. run Hamilton pipelines on different orchestration platforms). You could imagine compiling to airflow pipelines, running hamilton on metaflow, compiling to a vertex pipeline etc...

2. Adapters to load data from/save data to external providers. E.G. logging data quality to whylogs, loading data up from snowflake, saving a model to mlflow...

We're actively working on building this out though -- have some partners who hare helping build out use-cases!

Ola, I'm the founder of windmill [1] which is an OSS orchestration platform for scripts, including python, and I'm pretty sure one could compile an hamilton dag to a windmill openflow very simply (we use a mounted folder in `./shared` to share heavy data across steps). Right now, one can do ETL using polars on windmill but i'd love to have an even more structure code framework to recommend.

We should chat!

[1]: https://github.com/windmill-labs/windmill

Nice! Yep that's part of our vision to "compile onto", i.e. generate code for, frameworks such as yours.

Happy to sketch something out - want to join our slack to chat https://join.slack.com/t/hamilton-opensource/shared_invite/z...?

Reply on news.ycombinator.com