Temporal raises $103M in Series B funding
theregister.com
theregister.com
I say this as someone who loves using Temporal and see a big future for them.
Temporal is a managed state-machine. If you're familiar with K8s, you can think of it as a dedicated reconciler. Plain and simple.
Do you want a resource to go from one state to another (i.e. user orders food -> food is delivered) regardless of transient errors and other hiccups? Temporal handles the plumbing of driving your resource to it's desired end state.
Do you have a workflow that has multiples steps? Can anyone of those steps fail for whatever reason and you'll need to figure out how to do retrying/cleaning up? Temporal handles that.
Essentially you can write your business logic without the worry of including the (often) boilerplate retry, cleanup, scheduling, etc logic. If you're familiar with K8s reconciliation and how it retries failed operations again and again until it reaches the goal, it's much like that.
Temporal can do much more than that, but that's its bread and butter and I'm trying to describe it simply so that people can understand. It is a bit of a paradigm shift and has a bit of a learning curve when you first start using it.
But like others have said, it's very powerful.
i'm actually interested in a longer form "Temporal vs K8s" comparison - can make a case for Temporal atop k8s, or Temporal replacing k8s (already had a couple customers talk to us about this). It's a nuanced story to tell and I'm not sure if it has a conclusion.
Additional coverage for more info:
- https://www.geekwire.com/2022/temporal-is-a-unicorn-develope...
- https://www.theregister.com/2022/02/16/temporal_103m_funding...
- https://siliconangle.com/2022/02/16/temporal-raises-103-mill...
- https://dailyuknews.com/tech/temporal-raises-103m-in-series-...
- https://finance.yahoo.com/news/temporal-io-raises-103-millio...
- https://www.computerweekly.com/blog/CW-Developer-Network/Tem...
- https://www.indexventures.com/perspectives/everything-is-a-w...
(EDIT: links got fixed) For clickable links, remove the leading spaces so it doesn't get formatted as plaintext, and make newlines double to avoid them all being on one line
The biggest pain point is still deploying and configuring a production environment. I started suspecting it was so awful and poorly documented to push us to using the cloud offering, but then I couldn’t even find a link to sign up for cloud.
real answer is it's just really hard. even for us, much less the general developer audience. i wrote the docs we do have, and yeah it shows and I'm not proud of it. Temporal's initial demographic was entirely self sufficient in terms of devops knowledge to take the helm charts and docker stuff we have and translate it to their environment, and read source code/talk to us to fill in any gaps. But as we grow then we'll have to provide more and more rails for people who don't do that stuff.
tldr we hear you and a big chunk of this money will go toward both better docs and training for both open source and cloud users.
I ended up writing a set of AWS CDK constructs internally to provision clusters via ECS + EC2, with best practices baked in, but knowing what the best practices are is a little more difficult. I would prefer not learning through getting burned :)
Being able to model business processes--some of which may take multiple _years_, and require human approval/rejection steps--using procedural go/java/js, and knowing that once that function starts executing, it is basically indestructible (barring the complete destruction of temporal's backend database), is just really cool. Being able to outsource a lot of the complexity involved in using event sourcing to manage distributed transactions frees up our developers to worry about just the high-end business logic.
Congrats to the team! I'm excited to see Temporal gaining some more traction.
Temporal uses/implements/helps implement event sourcing easily?
in other words when you use us you get the main benefits of ES without the downside of having to code it up yourself, which is a common pitfall of homegrown ES systems.
If I DID want to get access to the event sourcing mechanisms under the hood - can I?
My hypothesis is that I can use Temporal to implement high quality DDD pipelines that follow CQRS - having explicit access to the event sourcing mechanisms would be fantastic!
Otherwise, it's an issue as most DBs can be argued to use some form of event sourcing mechanisms under the hood but they arn't built to be exposed outside the DB - hence an event sourcing mechanism has to be reimplemented (hence pitfall of homegrown ES systems).
What are your thoughts?
i try to say "we use event sourcing under the hood" rather than "we do event sourcing for you" for this reason - if you use us and expect the same level of control as homegrown youre gonna be disappointed
What's the design pattern you recommend Temporal users use to implement multiple consumers?
Usecase: Handle a large, spiky backlog by dynamically sharding the consumer input, starting and putting multiple consumers to work until the backlog is within nominal specs.
Eg: Shipping orders during a sales event. Each S&H order takes the same response time but multiple S&H orders can be parallelized and there are a few hundred thousand orders pending. Of course, S&H order can fail, in which case they are restored to the pending state after a timeout or loss of a lock
every temporal workflow goes into an assigned task queue, and Temporal handles distributing/load balancing to multiple workers polling that queue. you can have 10000 orders coming in simultaneously to 1 task queue being processed by 5 workers for example and Temporal would register heavy load but would still process through all that work in due time. the beauty is that when you write workflows you dont have to worry about acquiring locks or whatever, just write as though you had one durable long running process per order. its very freeing.
this is not to say it does everything, ie Temporal is not a replacement for a true pubsub model. we just described N workers processing 1 type of workflow initiated 10,000 times, but its not designed for 1 event type initiated 10,000 times that are reacted to by N types of processes that are supposed to be completely decoupled.
At first it looks easy to use any pub/sub or DB to create a job system, but there are so many edge cases, I'm not even talking about scaling etc ... just having a solid job system that properly handle retries, error, durability, timeout etc ...
Every 3rd party dependency is a complexity you must manage.
Every in house system is a complexity you must manage.
Nothing is free.
and on the Temporal side, not B2C directly, but serving large B2C. We now can cite Snap as a public customer - every Snapchat story runs through us (and yeah, theres not infinite spend for orchestration either so make of that what you will). and we have bigger users than Snap.
but we could add the ability for workflows to subscribe to kafka topics in future? no promises on the exact level of abstraction we’ll land on but its definitely been discussed
Temporal provides a unified backend for automatically managing implicit application state that is normally stored in transient queues, databases etc. Furthermore, Temporal does this without explicitly requiring the developer to think about and manage the state themselves. This means developers spend way more time building stuff that actually matters, and less time writing buggy reliability code.
I personally find the best way to explain it is with an analogy. Back in the late 90s many developers built applications with C and therefore had to manage their own memory. For a long time, this was not wasted effort as it was the only real option. But then, Java came around and offered an experience where developers didn't have to manage their memory. And for the majority of apps, the performance and capabilities of Java were more than sufficient. At this point, writing the average application in C meant you were doing a serious amount of undifferentiated work. Furthermore, most developers weren't that great at memory management so choosing to do it by hand meant more work for a worse result.
The value proposition of Temporal is nearly identical, but instead of manually managing memory with C, developers are manually managing state using queues, CRON services, databases and more. The result is a bunch of time spent doing undifferentiated things that a computer would have done better anyway.
Lib/framework to sync some state across some applications?
Temporal is a workflow engine for managing distributed state. It abstracts away queues, databases, schedulers, state machines, load balancers/gateways and makes it so you don't have to be a distributed systems expert to get this right.
To me there are four levels of appeal:
1. standardized, declarative system for timeouts and retries <-- most users start here
2. event sourced, highly fault tolerant internal architecture ---> we even like the term "fault oblivious" - when you write workflow code you can assume that it is robust to downtime in external APIs, or Temporal Server or Temporal Workers, the thing just keeps going until it hits a timeout or an actual application failure!
3. idiomatic SDKs for "workflows as code" -> no need to learn some JSON or YML based DSL, use all familiar software tooling
4. horizontally scalable architecture for every part of the system (key differentiator vs handrolled systems... although i'm not saying you should prematurely scale of course, just saying this architecture lets you scale without replatforming)
Some projects that we get compared to (although of course we're not 1:1 competitors): Apache Airflow, AWS Step Functions, Argo Workflows, Sidekiq, BullMQ
My 7 min intro: https://www.youtube.com/watch?v=CQhUL5RogXI
Architecture principles (23mins) https://docs.temporal.io/blog/workflow-engine-principles
Full blogpost: https://www.swyx.io/why-temporal/
The use case is basically any kind of temporally-distributed workflow that you might use something like Airflow for, or else otherwise hack together with some combination of event buses, temp tables, and cronjobs. Its main charm is that you define the logic in code rather than templates, so you can intermix business logic directly into the workflow.
ctrl + F to here "Temporal is a new kind of platform. It’s not a database, cache, queue, or a means to run your code. It’s a runtime for distributed applications that transparently manages their state at any scope and scale. By externalizing state to a generalized platform like Temporal, application teams get to outsource the majority of cross-cutting concerns presented by cloud applications. In doing so, applications become" if you dont want to read the whole thing
From my take, an interesting way of dealing with scalable workflows and batch jobs. But I need to read more into it.
It's like computer hibernation, but for a process that runs on a whole distributed system instead of on a single computer. As programmer of that process, you just focus on writing your business logic with the assumption that each step will eventually complete, and not have to worry about power loss/network loss/machine loss while your process is running. This way, it's much easier to reason about and focus on the thing you care the most: your business.
From what I know two people at Netflix tried to pitch and even implemented it in their projects. The rest of the company still uses other things and they have their own similar tool. Seems like false marketing to me.
I reached out to the guy trying Temporal at Netflix, apparently he never tried Conductor first. Big company problems I guess.
It will be interesting to see the future of Conductor given the Temporal efforts.
EDIT: According to https://blog.amplifypartners.com/our-investment-in-temporal/, Temporal was founded by 2 engineers at Uber who created an open-source orchestration engine called Cadence and then later decided to found Temporal.
I guess, AWS gave up too soon on Maxim's vision.
If my SQL & PHP backend serving a basic web API to a React storefront is struggling to handle customer's states (carts, lists, orders, etc) how is this going to help?
What are my engineers going to actually do?
To your point, the complexity of a SQL/PHP application is significantly smaller, but, the value proposition of Temporal is probably still there: If any part of the checkout process fails (e.g. charging the credit card, successfully triggering the order after they've charged the credit card, timing an email if there is an abandoned cart, handling the moving pieces of a return). These pieces are either handled manually by a human or you've got a decent amount of business logic in place to handle each of these edge cases individually.
The value-add of Temporal—in this case—would be that it would keep track of where the customer/order was in the overall flow of ordering something and pick up where it left off in the event that something went wrong.
when you are handling your customers' states, every purchase is a long running process, both from an ordering side and a fulfilment perspective. Work needs to be done asynchronously, or you need to wait for some condition to be met to proceed, etc. Temporal makes it so that you don't have to glue together a bunch of schedulers and queues and databases and ad hoc state machines to make that work, make it traceable/debuggable, and make that scale.
Your engineers would be able to translate your business requirements directly into workflow code (often just a function) and be able to version control, test, migrate, lint, etc, rather than have that logic spread out over a bunch of your code and infra structure. they would then register it with their Temporal workers, and then invoke/signal those workflows from your php/node/whatever application code.
i did a demo recently of what its like to translate product specs to a workflow, like @rrix is saying in another comment, its pretty fun! https://www.youtube.com/watch?v=2pxZgGhT-Xo
With event based system I would just set order=canceled and then for example cron to deliver stuff via email would not pick it up.
example usage in a billing workflow https://docs.temporal.io/docs/typescript/subscription-tutori...
I wonder how such long running workflows can be updated due to business requirements change. Say we now want to have longer trial period (so something that is not externalized via activity, like email wording for example).
https://docs.temporal.io/docs/typescript/patching
its not the simplest thing to do, and we have plans to make it simpler, but patching long running work while it is still running is inherently tricky. the good news is that at least we have a versioning story (and the failure mode is stopped progress) rather than handrolled systems which basically have “deploy and pray” strategies
Without Temporal, you store the state of the cart in the DB, load it when the app interacts with your backend, run some business logic and serialize state back into the DB.
With Temporal, first of all, there's no DB. The entire flow is modeled in a single piece of code. Your Workflow can listen to user signals to update the cart, queries to get the cart state, schedule durable timers to remind a user that their cart is abandoned after days and months.
I'd imagine if anything it makes databases _easier_ to reason with, because you could shard by workflow steps or something.
Here is a more detailed answer from Temporal forum: https://community.temporal.io/t/why-use-temporal-over-a-comb...
but yeah I would never say that we're for everyone. depends on your knowledge, needs, usecases!
Datadog used Temporal to automate database upgrades. Took a very manual, very unscalable process and automated it, got a lot more confidence in their processes.
How about this case study? https://temporal.io/descript
Descript used Temporal for more reliable, debuggable audio transcription. Took a failure prone process core to the company that was having 1 production outage a week to ~0.
I meant more specific coding examples of how it is used in code and why. Is it similar to any existing frameworks?