Is there any love for the Argo [1] project suite (Workflows, Events, CD) for this type of use case? I haven’t tried it out myself yet however it does look interesting.
Argo is another over-engineered "CNCF" thing trying to ride the Kubernetes hype train. It's all "eventually consistent", which makes it extraordinarily difficult to see when any particular thing actually happened. Is my code deployed? Who knows, Argo is "syncing".
Check out these great docs: https://argoproj.github.io/argo-workflows/rest-api/
> API reference docs :
> Latest docs (maybe incorrect)
> Interactively in the Argo Server UI.<https://localhost:2746/apidocs> (>= v2.10)
Yes, that is a localhost URL on their website.
;)
If you’re looking for something that’s a bit more high level and friendly to expose directly to your data team (data scientists/data engineers/data analysts) you can check out https://github.com/orchest/orchest
You can think of it as a browser UI/workbench for Argo scheduled pipelines. Disclaimer: author of the project