I appreciate that. Thanks for being available on hackernews and helping relay feedback.
72 karma · joined November 8, 2016
With the new pricing, I can't choose to use GKE instead of app engine/cloudrun and get the same availability (by this I mean multiple zones) without having to pay for both the nodes and the new control plane cost. Those managed products run over multiple zones in a region. It's disappointing that even just one multi-zone cluster is charged. I'd be very happy to see you include at least a single multi-zone cluster control plane in the free tier.
Edit: Additional comments: You could limit the number of nodes in the always-free-tier clusters. Above n nodes the free tier clusters aren't free.
With the new pricing, I can't choose to use GKE instead of app engine/cloud run and get the same availability without having to pay for both the nodes and the new control plane cost. Those managed products run over multiple zones in a region. It's disappointing that even just one multi-zone cluster is charged.
I believe the community is keen to engage with you on this based on the comments in this thread. If your team would like to talk to a disappointment (very small but hoping to grow) customer I'd be happy to jump on a call. I hope others here would be happy to do the same.
I was looking through the css of the 'abbey' page https://demo.ck.page/abbey and noticed a problem: the spinner doesn't show correctly on the submit button. More specifically, it does show, but the word 'Subscribe' is over the top when it should be hidden, so you can't see the spinner properly. The problem is an easy fix, it's just a small misalignment between the css selector and html structure :)
I haven't seen the QR code design on your homepage before. Did you custom make it yourselves? If so, could you share any links on how to do that? I have a QR code education app idea and I'd like to explore if it's possible to make more visually appealing branded QR codes :)
When you hoping to have git restore in the official git release?
Would you mind giving the current commands to achieve the three things you just listed?
That way I learn the future restore command along with the current approach and you description. T
For anyone interested apparently there's a free MOOC in may: https://www.kadenze.com/courses/introduction-to-programming-...
And a TED talk: https://www.ted.com/talks/ge_wang_the_diy_orchestra_of_the_f...
In the golang community for example Google has recently decided to build and officially support a language server (called 'gopls'). How it works: You build a language server once and get features like typecheck errors, auto complete, go to definition, code peaking, documentation on hover, etc... in all editors/IDEs that support the protocol.
LSP Website: https://microsoft.github.io/language-server-protocol/ Recent golang talk: https://youtu.be/bNFl7HcyDao?t=354
Workflow engines like Cadence essentially work by letting you write regular-looking procedural logic for your workflow. This looks and feels very much like writing an async function with async-await in javascript or C#.
The state in your workflow is then implicit in your code instead of explicit. Here's what I mean by that:
Usually you would explicity serialise your state between each incoming event and do an atomic-compare-and-set operation on an external database to store the new state.
For a new incoming event: (1) fetch the state, (2) marshal into an object in your programming language (ie a java class or golang struct), (3) given the current state, process the event and perform any external actions like sending an email. These external events need to be ok with being done with at-least-once semantics. Update the state object ready for the next incoming event. (4) serialise the state, (5) store the state in the database (atomically update with a compare-and-set operation). (6) Repeat on each event. Do everything with at-least-once repeat-on-failure semantics.
In a workflow engine like cadence, what is persisted to the database is the entire history of events instead of a single state object as described above.
In Cadence the code you write looks very much like async-await style code in languages like javascript or C#. The workflow logic is in some sense an async function that pauses at await statements and picks up again where it paused when the next event comes in.
Remember that cadence stores the entire history of events for a workflow. It does this so that it can rerun the workflow from the beginning, this time with the new incoming event on the end of the history.
Notice that you need to be careful about your workflow being deterministic.
Optimisations: (1) it knows when it is replaying already-seen-events and doesn't redo external events such as sending emails. (2) it tries to resend events to the same worker node each time. It caches events at worker nodes. (3) at the macro level everything is highly-available and repeat-on-failure-with-backoff to ensure progress and at-least-once-semantics. (4) it supports repeating workflows and child workflows (5) monitoring, tracing, other things you'd expect (6) etc
Importantly: notice that there is still in some sense a single state object. The state is just implicit: it is deep down in the internal state machine of the language you wrote the workflow function in (java, golang). Instead of serialising state such as 'time-since-last-email' in a state object to a database... you have 'time-since-last-email' as a local variable in the scope of the workflow function. Similarly your programming language is tracking the call stack and current execution position of the function... normally you'd keep track of progress through the workflow in the state object and condition on this state when receiving a new incoming event.
Thinking about state as explicit (state object approach) versus implicit (replay-history approach) helps me when thinking about cadence and similar workflow engines.
....................................................... Thanks for reading so far. I'd love to hear from users of cadence at uber or elsewhere:
(1) why do you choose to write workflows with implicit state (by replaying history) instead of storing the state explicitly as a serialised state object in the database?
My guess: developer productivity of writing and maintaining the workflows. Having a common approach and single observable system for many different workflows.
(2) how do you reason about long-running workflows where the business logic needs to be updated? Would this not be much easier if the state object (say a serialised protobuf) was stored explicitly in the database?
(3) wouldn't non-determinism be much easier as well if you stored the state explicitly?
If anyone could help me understand the following I'd appreciate it...
I understand that the event history is cached at worker nodes and the whole history of events is only delivered to workers if needed (ie if out of cache) and that normally cadence manages to deliver events for the same workflow to the same worker.
My question relates to what exactly happens within a single worker process.
On each new event, does the worker process loop through all events in the history - starting from the beginning - in order to get back its internal state, and then process the new event, and then shutdown ready to repeat this for the next event.
Or... does cadence keep the internal state around by keeping the goroutine alive waiting on a channel midway through it's workflow logic and waiting for the next event to continue execution.
Thanks