HNHacker News
TopNewBestAskShowJobs

KraftyOne

1,457 karma · joined June 20, 2019

Hello! I'm Peter Kraft. I love databases and distributed systems and recently co-founded DBOS to help developers build reliable software effortlessly.

Website: http://petereliaskraft.net/

submissionscomments
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
Thanks for the great questions!

1. Yes, currently versioning is either automatically derived from a bytecode hash or manually set. The intended upgrade mechanism is blue-green deployments where old workflows drain on old code versions while new workflows start on new code versions (you can also manually fork workflows to new code versions if they're compatible). Docs: https://docs.dbos.dev/java/tutorials/workflow-tutorial#workf...

That said, we're working on improvements here--we want it to be easier to upgrade your workflows without blue-green deployments to simplify operating really long-running (weeks-months) workflows.

2. Could I ask what the intended use case is? DBOS workflow creation is idempotent and workflows always recover from the last completed step, so a workflow that goes "database transaction" -> "enqueue child workflow" offers the same atomicity guarantees as a workflow that does "enqueue child workflow as part of database transaction", but the former is (as you say) much simpler to implement.

KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
The main difference is that this is a library you can install and use in any application anywhere, while Durable Functions is (as I understand it) primarily for orchestrating serverless functions in Azure.
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
Haha yes, one thing you can use this for is "long waits" or "long sleeps" where a program waits hours or days or weeks for a notification (potentially through server restarts, etc) then wakes up as soon as the notification arrives or a timeout is reached. More info in the docs: https://docs.dbos.dev/java/tutorials/workflow-communication
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
We'll be releasing self-hostable Conductor (the web UI) soon--stay tuned!
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
Fully compatible--see this discussion on Github: https://github.com/dbos-inc/dbos-transact-ts/issues/1115
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
.NET support is something we're considering, but aren't actively building right now.
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
One thing we're looking at right now is what it would take to support Clojure or Kotlin.
KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
We actually wrote a blog post recently on why we chose Postgres! https://www.dbos.dev/blog/why-postgres-durable-execution

There's no technical reason why this couldn't be done with another database, and we may add support for more in the future (DBOS Python already supports SQLite), but we're not working on it right now.

KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
Thanks for the great feedback! Yeah, for isolation we recommend each service have its own system database and communicate via clients (so service A starts a workflow in service B by creating a client and calling "client.enqueue").

How could we make this experience better while keeping DBOS a simple library? One improvement that comes to mind is to add an "application name" field to the workflows table so that multiple applications could share a system database. Then one application could directly enqueue a workflow to another application by specifying its name, and workflow observability tooling would work cross-application.

KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
Our primary focus is Postgres. DBOS Python recently added SQLite support, we'll add this to other languages if it proves popular, but no current plans for MySQL.

That said, while DBOS requires Postgres for its own checkpoints, it can (and often is) used alongside other databases like MySQL for application data.

KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
There's an observability and workflow management UI: https://docs.dbos.dev/java/tutorials/workflow-management

You can view your workflows and queues, search/filter them by any number of criteria, visualize graphs of workflow steps, cancel workflows, resume workflows, restart workflows from a specific step--everything you'd want.

Currently, this is available as a managed offering (Conductor - https://docs.dbos.dev/production/self-hosting/conductor), but we're also releasing a self-hostable version of it soon.

KraftyOne··on Show HN: DBOS Java – Postgres-Backed Durable Workflows
DBOS Python works with gevent out of the box (sync DBOS uses Python threading APIs and psycopg3 that gevent monkeypatches).

Have you run into any issues using DBOS Python with gevent? Please let us know!

KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
DBOS also has a full-fledged workflow visualization and management UI: https://docs.dbos.dev/golang/tutorials/workflow-management
KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
Yes, there's a full workflow visualization/management interface (not embeddable though): https://docs.dbos.dev/golang/tutorials/workflow-management
KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
There are DBOS libraries in multiple languages--Python, TS, and Go so far with Java coming soon: https://github.com/dbos-inc

No Rust yet, but we'll see!

KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
We may be a small startup, but we're growing fast with no shortage of production users who love our tech: https://www.dbos.dev/customer-stories
KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
The durability guarantees are similar--each workflow step is checkpointed, so if a workflow fails, it can recover from the last completed step.

The big difference, like that blog post (https://www.dbos.dev/blog/durable-execution-coding-compariso...) describes, is the operational model. DBOS is a library you can install into your app, whereas Temporal et al. require you to rearchitect your app to run on their workers and external orchestrator.

KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
Yes, in any durability framework there's still the possibility that a process crashes mid-step, in which case you have no choice but to restart the step.

Where DBOS really shines (vs. Temporal and other workflow systems) is a radically simpler operational model--it's just a library you can install in your app instead of a big heavyweight cluster you have to rearchitect your app to work with. This blog post goes into more detail: https://www.dbos.dev/blog/durable-execution-coding-compariso...

KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
Yeah, queue priority is natively supported: https://docs.dbos.dev/golang/tutorials/queue-tutorial#priori...
KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
1. We also have support for Python and TypeScript with Java coming soon: https://github.com/dbos-inc

2. There are built-in APIs for managing workflow recovery, documented here: https://docs.dbos.dev/production/self-hosting/workflow-recov...

3. We'll see! :)

KraftyOne··on Dbos: Durable Workflow Orchestration with Go and PostgreSQL
The specific claim is that workflows are started exactly-once in response to an event. This is possible because starting a workflow is a database transaction, so we can guarantee that exactly one workflow is started per (for example) Kafka message.

For step processing, what you say is true--steps are restarted if they crash mid-execution, so they should be idempotent.

KraftyOne··on I solved a distributed queue problem after 15 years
Kafka is great for streaming use cases, but the big advantage of Postgres-backed queues is that they can integrate with durable workflows, providing durability guarantees for larger programs. For example, a workflow can enqueue many tasks, then wait for them to complete, with fault-tolerance guarantees both for the individual tasks and the larger workflow.
KraftyOne··on I solved a distributed queue problem after 15 years
Observability is a big advantage, another advantage (in the context of DBOS specifically) is integration with durable workflows, so you can write a large workflow that enqueues and manages many smaller tasks.
KraftyOne··on Build durable workflows with Postgres
That's interesting, I'll take a look! I had always thought of it as an Azure-only thing.
KraftyOne··on Build durable workflows with Postgres
Like Inngest and Restate, DBOS provides durable workflows. The difference is that DBOS is implemented as a Postgres-backed library you can "npm install" into your project (no external dependencies except Postgres), while Inngest and Restate require an external workflow orchestrator.

Here's a blog post explaining the DBOS architecture in more detail: https://www.dbos.dev/blog/what-is-lightweight-durable-execut...

Here's a comparison with Temporal, which is architecturally similar to Restate and Inngest: https://www.dbos.dev/blog/durable-execution-coding-compariso...

KraftyOne··on Build durable workflows with Postgres
Yeah, I think in that case you should have auto-scaling DBOS workers all pulling from a queue and a FastAPI backend using the DBOS client to submit jobs to the queue.

Queue docs: https://docs.dbos.dev/python/tutorials/queue-tutorial Client docs: https://docs.dbos.dev/python/reference/client

KraftyOne··on Build durable workflows with Postgres
I'd love to learn more about what you're building--just reach out at peter.kraft@dbos.dev.

One option is that you have DBOS workflows that schedule and submit jobs to an external worker app. Another option is that your workers use DBOS queues (https://docs.dbos.dev/python/tutorials/queue-tutorial). I'd have to better understand your use case to figure out what would be the best fit.

KraftyOne··on Build durable workflows with Postgres
Would love to learn more about what you're building--what problems or parts of your system would you solve with Dagster vs DBOS?
KraftyOne··on Build durable workflows with Postgres
We wanted to make workflows more lightweight--we're building a Postgres-backed library you can add to your existing application instead of an external orchestrator that requires you to rearchitect your system around it. This post goes into more detail: https://www.dbos.dev/blog/durable-execution-coding-compariso...
KraftyOne··on Build durable workflows with Postgres
Both do durable workflows with similar guarantees. The big difference is that DBOS is an open-source library you can add to your existing code and run anywhere, whereas Durable Functions is a cloud offering for orchestrating serverless functions on Azure.
← PreviousPage 2 of 4Next →