rubenfiszel

Some people will argue we are open-core because almost everything is open-source (AGPL) but our sso and enterprise plugins are under a source-available proprietary license.

The reason that we didn't split them out of the codebase is that it's harder to maintain and would require to load the plugins. We didn't want to waste time on that when we had so much to build.

Not only it can cover those cases, you will likely be pleased with the advantages of such system (the automatic GUI creation is just one of those).

The binary itself doesn't sandbox your processes unless you're in the nsjail mode. Hence, if you either use the binary raw (without docker) or mount the filesystem to be available to your container, it can run anything that is available on the filesystem.

It's seconds. You can find all the detailed numbers there: https://www.windmill.dev/docs/misc/benchmarks/competitors

When you tried, we didn't have s3 integrations nor restartable flows. We will present all of this in day 5 of our launch week and it might be a good time to revisit.

I agree we were not a good fit prior. I think we would now compare favorably as we will offer excellent ergonomics for data processing, leveraging polars, duckdb, and other OLAP libraries to their full extent.

We could add support for Rust, and it would work the same as go, do a built step prior, and then cache the binary. If we have enough demand for it, we will add it (we are a small team and can only prioritize what's popular unfortunately).

On a personal note, I write a lot of rust (and love it) but using rust for writing non-backend type of stuff is not something I would do myself. I'm more of a believer of hybrid like polars that optimize the compute in Rust but provide a compatible sdk in python.

you can always run compiled programs from bash as binaries (but yeah it's not as great as an experience than being able to run interpreted languages directly).

Yes it's too slow for HFT, but anything would be too slow for HFT unless it's custom built. You likely want a streaming engine or even hand-optimize all your event handlers to shave-off any nanoseconds.

Yes, windmill would likely be a great fit for that as long as you're willing to do a little bit of coding.

For instance, we do not have a calendar integration per se but can manage your oauth tokens towards any service (including gcalendar) and allow you to write code using their official sdks.

For spreadsheets, you can build a similar interface in windmill using aggrid.

It seems you also need approval steps and have your workflows wait for events, and that's built-in in our workflows.

The critics is more than fair and our positioning is not very clear.

We are doing a developer platform for enterprise to build internal software, including apis, workflows, background jobs, and UIs using code, but only where it matters. We happen to be quite decent in all aspects, with a focus on performance, and we have an active community of users and customers that are mostly developers and hence have plethora of feedback and feature requests, so we're happy to oblige.

By being at the right level and trying to expose the code as much as possible, we can be great generalists without sacrificing on the quality. Python, Typescripts, Go, Bash and all the query languages are where all the complexity is and capabilities come from. They are well developed languages with great ecosystem. We merely provide a way to run those language in a system fit for production at enterprise scale.

You're right, I apologize. Truth is I wrote everything in vscode markdown and really wanted to finish the article before going out for dinner and I ended up skipping it. We fixed everything

So I agree that for most jobs taking already more than 300ms, a 50ms per job will not be perceivable anymore. What matters is the DX and even though I'm very proud of our performance, a lot of our work is focused on providing a consistently great developer experience in all aspects (vscode extension, instant deployment, etc, etc).

However, there is one big benefit to being performant which is that you can use the same stack for performance sensitive jobs such as event-streaming use cases. That remove the duplication of infrastructure. The other benefit is that it improve the developer experience overall with fast previews.

Curious, what's the difference between an article and a blog post ?

Looks cool and thanks for sharing. Founder of windmill.dev, an open-source, extremely fast workflow engine to run jobs in ts,py,gosh whose most important piece, the queue, is also just rust + postgresql (and mostly the FOR UPDATE SKIP LOCKED).

I'd be curious to compare performances once you guys are comfortable with that, we do them openly and everyday on: https://github.com/windmill-labs/windmill/tree/benchmarks

I wasn't aware of the skip B-tree splits and the REINDEX CONCURRENTLY tricks. But curious what do you index in your jobs that use those. We mostly rely on the tag/queue_name (which has a small cardinality), scheduled_for, and running boolean which don't seem good fit for b-trees.

We do support pwsh :)

We have built an open-source generic workflow engine to run arbitrary scripts (https://windmill.dev) with a vscode extension to build the yaml using a low-code builder and each individual script in their dedicated python/ts files so you get your full editor assistants https://youtu.be/aSOF6AzyDr8?t=116

One of the area we are expanding next is a github app so you get exactly the same UX as github actions but running windmill workflows on your windmill workers.

One way to use raw python/javascript/bash on top of a postgres scheduler is to run a single container of https://github.com/windmill-labs/windmill. It only requires a PG connection and does not require a pg extension. It implements a scheduler that is implemented on top of PG and can run any of the aforementioned language in a very practical and observable interface.

Looks like a rust alternative to temporal using wasm as a runtime. Love it!

Founder of windmill.dev here which is another durability engine in Rust except it's a lot less elegant since we split our workflows into well defined steps in python/typescript/go/bash and can resume from incomplete steps only by restarting from the last step and storing the result of each step forever in our postgres db (using jsonb). The use-cases are clearly different and I can see flawless being so lightweight that you could use it to model UI flow state and scale it to millions on a small server as pointed out by the site.

This is fantastic, hopefully one day rust will power all distributed systems.

The wonders of HN strikes again. Thank you for this amazing piece of technology that is nsjail. Nsjail is very core to our security, our multitenant would be so slow without it and I think we're one of the applications that leverage it in a way that showcase nsjail to its full extent (as in, we beat containers/firecracker cold starts by a fair margin while keeping most of their benefits). That's one of the reason we're order of magniture more efficient than Airplane that uses fargate under the hood. I would love to chat if you had time, my email in my profile.

If you are looking to self-host a scalable backend that runs arbitrary code in python/typescript/bash/go with optional sandboxing using nsjail like figma, nsjail is what we use as isolation layer at https://windmill.dev (Open-source alternative to Retool/Airplane)

(Our python nsjail config for instance: https://github.com/windmill-labs/windmill/blob/main/backend/...)

Palantir is a particularly bad example, it being american. France care a lot about sovereignty and would never allow (and for good reasons) an external entity to have that much control. It probably did ask for a contractor on an "appel d'offre" to build this so it's the same but with a french actor.

We do a bit the same at windmill.dev except for python and typescript and we parse the parameters directly from the signature without needing to use macros/annotations. I wonder if the same could not have been done here where the macro would have been as a build processor.

Founder of windmill here.

You can, and that's the main workflow for most users, use your code editor and version control with windmill, and you can access your database and existing code easily. It works exactly the same way as trigger where you sync and deploy from your existing code repo.

Inngest and trigger are event-driven workflow engines while windmill is a more traditional workflow engine (think modern airflow) where the flow/graph is defined statically in a low-code UI (although can be generated dynamically) while the steps are code-centric. We also include a UI builder (similar to Retool) and have an heavy focus on running on your own infrastructure (k8s with helm or docker-compose) and include complete observability for heavy jobs (streaming logs in real time) as well as the ability to use hardware acceleration since we use your raw nodes. Windmill also include workers management and is polyglot, You can run typescript, but also go, bash, and python, and write queries for bigquery, snowflake, and postgresql without having to wrap them in typescript.

So I would say we are less focused on integration to external APIs and more focused on enterprise use cases, for critical and heavy background jobs that require long and complex workflows (for instance that may require approval steps). But I think you guys are also working on background jobs and I haven't seen it so hard to say.

Good luck to you all. I think for the people that know well both frameworks, it would be easy to discern when windmill or trigger is a better fit and I agree the use-cases are different.

you should try windmill.dev, semi-technical folks can use the webeditor and the built-in versioning while more senior folks can keep their IDE and version directly from github/gitlab.

The UI part is drag-n-drop;

Interesting post and would like to bring nuance as the founder of an open-source alternative to Retool and Airplane: https://github.com/windmill-labs/windmill

Our enterprise edition is clearly cheaper (between 2-5x cheaper based on customers migrating away from those) than our competitors but we bet on volume and lack of sales team to make it up for it in terms of margins.

There are pragmatic reasons for it, our community/oss edition is featureful and include SSO so there is a temptation for our customers to just roll with the CE and we cannot be too greedy.

One could argue that support should be sufficient since the jobs ran by windmill are critical, but the main strength of a product like this is reliability and since we achieve near 100% reliability, it isn't sufficient. We are not gonna intentionally make our product less reliable on CE.

The other reason in my opinion is deeply rooted with the nature of open-source: software cost nothing to replicate and it would be a shame to have users not benefit from the right tools just so the pricing can extract as much $$$ as possible.

So ultimately it comes down to the tension between being VC funded (and we are but to a much lesser extent that our prop alternative) and the pricing of open-source. I am deeply convinced that there is a compromise that satisfies everyone by having the different force in presence being kept-in-check. VCs want small seed companies to scale to become global companies; customers and users want to invest in a platform they know they won't have to regret later because of rent-seeking or lock-in practices. True OSS helps companies achieve global scale as long as their product is better (which make me agree with the substance of the post, being cheap is not enough) and also ensure that company can never or hardly employ dark patterns otherwise OSS fork of company's product will be too competitive against itself.

Before being a founder, I was among the most skeptical of OSS with VC backing and I really understand why someone would have a hard stance on it. But I came to realize that good software takes not only lots of hard work, but a focus and a dedicated team that is hard to find without a core team that has strong incentives to make something that people want. So I see the future being dominated by companies that fall somewhere in the following spectrum:

1. OSS products that are public utilities and there are enough needs or the project is interesting enough to have a strong core team that doesn't require VC backing

2. OSS products that are more commercially oriented and have values but wouldn't exist without VC backing

3. OSS products that were able to bootstrap themselves completely and are COSS but without VCs

And proprietary software to slowly become extinct as the world of software become more competitive everyday as the potential grow larger and the number of VCs that will fund open-source grow larger (and the ability to bootstrap those businesses also get easier).

Exactly! It looks and feel like normal JS and hides all the complexity behind it which is the print of great syntax.

This is amazing syntax. If you are looking for a self-hostable version of this, that can run deno but also python, go, and bash, with primitives similar to airflow and more (retries, cache, suspend, approval steps), check out windmill: https://github.com/windmill-labs/windmill

We use exactly this for windmill (OSS Retool alternative + modern airflow) and run benchmarks everyday. On a modest github CI instance where one windmill worker and postgres run as containers, our benchmarks run at 1200jobs/s. Workers can be added and it will scale gracefully up to 5000jobs/s. We are exploring using Citus to cross the barrier of 5000j/s on our multi-tenant instance.

https://github.com/windmill-labs/windmill/tree/benchmarks

you can also use open-source automation tools like n8n or windmill.dev, or github and gitlab workflows as mentionned above.

windmill.dev is a modern take on Huginn: optimal performance, great DX, scale to any cluster, work with python, typescript, go, bash, and flows that are more powerful.

n8n is great because their UX is simple and they have many well-made integrations, including an automatic way of setting up the webhooks in your github account for instance to trigger automation on changes to your repo.

Windmill is less about having many integrations, and more about being able to write real code-based workflows, running on your infra and with potentially long-running jobs that you can assign easily to bigger nodes. It's a compromise between n8n and temporal for the workflow part.

Anyway, n8n was a precursor in many aspects and a great inspiration for all of us building workflow engines.

If you do not know the product then maybe instead of saying that X is simpler/faster/better than Y it would be better/fairer to say "I don't know" or "from a look at their landing" ?