rubenfiszel

Open-source founder of another startup/open-source project with an ee piece [1]

Separating the repos is a HUGE hassle which is why you never see it anywhere. Doing it once is a small cost but maintaining is a huge burden on teams that are already too small. In practice it does not change anything so most projects will choose to suffer nit-picks.

[1]: https://github.com/windmill-labs/windmill

> Data pipelines (eg Apache Airflow) where the orchestration is handled by Rust but the high-level business logic is defined using JavaScript

That is ... exactly what windmill is https://github.com/windmill-labs/windmill

The orchestrator is built from scratch in rust using postgresql to store the state. The steps when in typescript are run in deno: https://github.com/windmill-labs/windmill/blob/main/backend/... and the transforms between the steps (piping outputs of any node to the input of any node) are actually javascript expressions run by deno JsRuntimes: https://github.com/windmill-labs/windmill/blob/f3ec9ca09dc37...

Windmill | Paris or Europe | Rust + Typescript Founding engineer | Full-time | Remote or hybrid | https://windmill.dev

Windmill is an open-source developer platform [1] to write and turn scripts into endpoints, workflows and UIs. It can be seen as an open-source alternative and cross between AWS lambda, n8n/temporal, Retool and Airplane.dev with an emphasis on performance. Deploy our helm charts on-prem and have a scalable infra and product to build all of your internal tools. We care a lot about correctness and performance at scale as our primary customer audience is enterprise (but many smaller startups and individuals use the fully free OSS version)

Hiring for one Rust + Typescript founding engineers to work on orchestration, workflows, and CLI.

Contact me at ruben@windmill.dev

[1]: https://github.com/windmill-labs/windmill

For the Windmill [1] project, we wrap deno to make it the backbone of a self-hostable infra to run scripts (like aws lambda), workflows (like temporal/n8n) and UIs (like retool). This release is amazing for a few reasons. We leverage enormously that you can define script with dependencies in the same file and provide a monaco editor backed by the deno lsp through websockets. So without deno, windmill would not be possible. We also support python too but it's a much slower as a language and make more sense for slow, long-running processes like ML.

> Improvements to npm and Node compatibility

Every little improvements to npm compatibility actually unblock a very wide range of npm packages to now 'just work'^tm. I would have been interested to get numbers of incompatible library but `npm:crypto` has been the biggest blocker for our users.

[1]: https://github.com/windmill-labs/windmill

We have built a cross between airplane.dev, retool and n8n. It's fully open-source: https://github.com/windmill-labs/windmill Compared to airplane.dev, our scripts are instantly deployed, run faster, and we are being deployed fully on prem air-gapped, and scale easily to large clusters. Our workflow builder is similar in features to a tool like temporal and can be used to run ETL on nodes with hardware acceleration. Last, our UI builder is more similar to retool so it feels similar to traditional BI tools to build dashboards.

Am I the only one to wonder why one of the greatest mind of this generation doesn't have an assistant to free his time for some more worthwhile use of it?

Thank you for open-sourcing especially under MIT license. I am digging through the codebase and finding a lot of interesting things.

I am building another open-source project that is also a self-hosted deno runtime written in Rust, Windmill [1], where we enable to build all of your internal tool and infra (endpoints, workflows, apps) based on scripts (deno, but also python, go, bash). Instead of running one http server continuously for your function, we run it on demand which has its own set of challenges.

We are doing something pretty naive right now: we create a fork and call deno run [2]. It's decently efficient (25ms for a naive script e2e). We are familiar with deno_core and use it in other places to run javascript directly, but for typescript, deno didn't expose directly the root entrypoint as a lib so we had to fork it [3] and are now gonna be able to do the transpiling to js AOT and save the fork for sub 5ms script execution.

We also want to make some functions togglable as high-performance endpoints and for those we would want them to be spawned as http servers to save the cold start. I'm gonna investigate thoroughly the codebase, and thank you very much for having shared it.

[1]: https://github.com/windmill-labs/windmill

[2]: https://github.com/windmill-labs/windmill/blob/main/backend/...

[3]: https://crates.io/crates/deno_cli_lib_windmill

For the need of windmill.dev (OSS retool + FaaS), we support deno and python. To provide that same experience that you're speaking of which is "python run myscript.py" without having to pre-install the dependencies, we wrote our own python parser in Rust which infer the PyPi packages from the imports (and then our workers download them if they are not cached). Here is the code from the parser: https://github.com/windmill-labs/windmill/blob/main/backend/... which we could carve out as a separate module.

We have clients solving this pattern using supabase (storing the client's credentials on supabase and then using RLS) and our open-source solution Windmill to solve this exact problem.

S22, solo founder, open-source infra + internal tools (windmill.dev).

As the other ones in this thread have mentionned, YC is what you make out of it. For the fundraising alone, it allowed me to raise - a non US solo founder - at very comfortable terms and on the financial aspect alone it would have been worth it. But most importantly, the hardest part to start a company in my opinion is to keep being motivated.

I was under the impression that surely a successful startup wouldn't have to go to any struggle and that founders were a special - genius-like - kind of breed. Being in YC opened my eyes:

- everyone struggle, relax and keep working

- the great people of this world are smart, but not crazy smart. They are crazy pragmatic, and crazy motivated, which you can be TOO

- YC founders are nice people, and they make for great friends to keep. Those people are the only one that will understand truly what you're going through.

YC was a transformational experience for me and I would recommend it to all.

EDIT: To give a more balanced view, YC was also stressful, and the expectations were the same, solo founder or not. Be prepared for an intense ride.

When we dev, svelte uses vite which has instant hot-module-reloading. You edit your code and see the change live.

The build itself takes around 2 minutes, but that's mostly due to monaco and tailwind taking the majority of the time to be bundled.

We have a pretty huge codebase [1] in svelte for Windmill, probably one of the biggest SPA on it. We are an open-source n8n/temporal + retool alternative so we are building both an advanced flow builder and an advanced app builder.

The article is on point. There is 1 caveat to animations that I'd like to add. Everytime we ended up using animation without the `|local` specifier, it broke completely our app, and that's an easy mistake to make. It should be the default imho. That's because svelte is updating the whole DOM instead of that specific component without it, and it waits for the animation to be observably over to re-render the DOM .

For reactivity, it works surprisingly well but for big state updates, and a redux style of having one big state to update, we ended up splitting the state subparts that were independent in reactivity because otherwise you end up very easily into a reactivity hell (reactivity, calling other reactivity). We also have a lot of pattern to keep in check the reactivity on nested objects such as using object comparison checks (using fastEquals) to make sure the input object actually changed and it was not a leaf node unrelated to this component that triggered the change.

Overall, with the small team that we are, we could NOT have build such a complex and blazing fast superapp without svelte. On the other hand, we had to become expert at knowing how svelte would actually compile to otherwise we would have been stuck early at using the reactivity bindings after a certain scale.

[1]: https://github.com/windmill-labs/windmill/tree/main/frontend

Not wanting to start an anti-apple thread but I meant dominant position in their closed garden. Is that an abuse or not, I'll let the litigators decide.

I am neither an Apple lover or an Apple hater. I think Apple produces extremely well polished products thanks to their vertical integration and have innovated in hardware pushing the frontier, especially with the ARM suites. However, they also have a fairly closed ecosystem in which they are trying to overprice things when they can abusing their dominant position. I would feel sad to contribute to its success but I also selfishly really need the best tool available.

I have been a thinkpad + arch devoted user for the last 10 years. I just want a nice ARM machine now and it seems the best option at the moment is Macbook Air M2 + Asahi. I do not know how to feel about it, maybe a bit of sadness, but I wish great luck to Asahi.

Also every blog they post are great read!

But in the end, they grew it to $1.2M ARR. That some very cost sensitive people are finding other ways is not much of a loss because it's very hard to get money from those people anyway.

We are building windmill.dev this way and fully open-source, I'm less worried about copy-cats than about the project not taking off because of too much friction.

Ola, I'm the founder of windmill [1] which is an OSS orchestration platform for scripts, including python, and I'm pretty sure one could compile an hamilton dag to a windmill openflow very simply (we use a mounted folder in `./shared` to share heavy data across steps). Right now, one can do ETL using polars on windmill but i'd love to have an even more structure code framework to recommend.

We should chat!

[1]: https://github.com/windmill-labs/windmill

docker compose up and you're gtg. We also have helm charts and it can also run from a single binary and a postgresql url. Everything is documented on windmill.dev

You might want to give a try to windmill. open-source, self-hostable FaaS and app builder similar to retool.

[1]: https://github.com/windmill-labs/windmill

For Bun on workflow builders, I'm currently working on making windmill [1] supports bun as an additional runtime to run the typescript scripts/steps. It's not the same as making the workflow engine run with bun (we use Rust) but I thought it might be worth mentioning :)

[1]: https://github.com/windmill-labs/windmill

This is correct up to a point (which is why npm ci exist at all), I agree that in general since you would have to mutate the package.json for that to happen, my point is moot (https://stackoverflow.com/questions/45022048/why-does-npm-in...) but npm install will overwrite the package-lock.json if they do not match.

It seems to me that the norm in the node world is to use ^ specifier to say anything that is non-breaking, and that `npm install` will not respect the package-lock.json, only `npm ci` does.

Wanted to shime in on this since we are big supporters/users of deno and building an OSS enterprise-grade platform for a self-hosted lambda that runs deno scripts among others [1].

I think this is unfair because npm has its own set of issues, most notably, if anyone were to sneak up a non obvious trojan horse into the dependency chain, it would take a few weeks to be detected and millions of servers would already be compromised. You can do this with deno too but the norm is to use pinned version and a deps.ts, it takes a bit more than just an innocent npm install to break things.

The real security for scripts is in the sandboxing, permissioning model and providing self-contained scripts that you can read and analyze in one go. So far, node doesn't offer that experience yet. Deno is the leader of innovation in that specific area. It was a bold move and that it is moving back to node compatibility has a lot more to do with network effect and first mover advantage than an inherently bad design.

Bun is likely to have made the correct move by focusing on performance and node compatibility from a business/adoption standpoints (and we will add bun to our supported env as a consequence), but from a security perspective, deno is still the cutting-edge.

[1]: https://github.com/windmill-labs/windmill

We've been very happy customer at https://github.com/windmill-labs/windmill, all of our docker builds are on depot and it replaced our fleet of github runners on hetzner :)

Congrats on the launch, as a growing and open-source startup ourselves, we are eager to switch to Lago as soon as our stripe fees are gonna become a PITA which I expect to be in the near-future. The product looks amazing and the API seems super well-designed!

I love deno so much and all of its ideas highlighted in the article (dependency management, security, performance) so much that I've build an entire framework around making scripting enterprise grade that is centered around deno support. The project is open-source [1], we even expose a web monaco editor and made the lsp support available using websocket and json-rpc. The goal here is to build enterprise-grade one-off script and workflows in deno, that can benefit from permissioned credentials, autogenerated UI (by parsing the main function parameters), a workflow engine similar to airflow/temporal, etc ...

We have a hub [2], centered around deno, that serves as our library of integration. An integration for us, is just a script that uses the right dependency to do an atomic action like fetching data or doing a POST.

We are betting big on deno, and are hoping with windmill to be the framework to make it enterprise-ready for other things than webservers (which most of the deno framework currently focus on).

[1]: https://github.com/windmill-labs/windmill [2]: https://hub.windmill.dev

You do not need the enterprise license to run this self-hosted, only if you want the commercial license version of AGPLv3 + support/SLA and some high scale plugins like our dependency cache syncing.

Your calculation is correct and I agree it is a bit of a steep pricing. Most companies have small scripts to run and we wanted to bill according to the value they were extracting of the product and do transparent enterprise pricing instead of the usual "contact us". Unfortunately, it is not a one-size-fits-all situation. For data heavy/IO bound workflows, we should bill per number of GB transferred rather than per computation. In any case, we are very open to do price capped contracts depending on your use case: ruben@windmill.dev

The biggest reason is to be able to deploy, as you mentioned, windmill as a single binary that you can then easily scale horizontally. The second one is deeply rooted to how it works, it is running fork of those scripts calling the interpreter of those respective languages. It has pros and cons but you can do interesting things like having workflows in different languages, first step in python, second step in go.

Overall it would have been possible to build this on top of Temporal but the biggest benefit of Temporal, which is idempotency/checkpointing is not something we could have offered at the level of abstraction that we operate, running arbitrary single scripts piped together using a result graph and a shared mounted folder. So why bother?

Trigger.dev looks like it would be amazing if you want a simplified temporal that is still 100% code based and exclusively in Typescript. If you are looking for a Zapier-for-developer experience, we took an alternative route at Windmill of allowing Python/Javascript/Go/Bash as steps but the workflows are still GUI based: https://github.com/windmill-labs/windmill/blob/main/imgs/win...

Despite them being GUI based, you can do everything that temporal enable, approval steps/suspending workflows until being reactivated by a webhook, branching, etc, etc... It's 100% open-source and self-hostable.

> I'm really looking for something that is gui first but as a drop down to code when you want. But inline code editor, let me save it to a git repo or similar.

That's like, exactly what we are, take a look at this screenshot for instance: https://github.com/windmill-labs/windmill/blob/main/imgs/win...

As for syncing code to git, we do it as well through our CLI and Github Actions: https://github.com/windmill-labs/windmill-sync-example