5,770 karma · joined October 22, 2012
Firebase -> Google -> Futurice -> Taktile
Now in Berlin.
But if the producer retries and the consumer does not respond with 200 until it has processed the message, no consumer side message queue is needed, the consumer can rely on the producer to reach at-least-once delivery.
In both cases (webhooks and /events) storage is needed producer side, so nothing consequential has changed, only with /events you need long lived TCP connections which ties up (e.g. you can't do this on FaaS endpoints)
Functions as a service are absolutely ideal for low frequency webhook receivers. SO SO SO cheap.
You do not need storage in the producer AND in the consumer, you just need a queue in the producer. Yes, even that is annoying, but the suggested architecture will still lose data if the long poller is down unless there is storage in the producer... so nothing significant has really been solved
1. bad actors can't hide B.S.
2. good actors have something they can point to that has 3rd party validation.
So for digital services where the source code is always available, my runtime dynamically loads the source code indicated in the URL each request. So you are free to go to the URL directly, and you can figure all the routing out in advance of data leaving your device (in theory).For the sake of moving fast, the 1st supported code repository the runtime can run serverside code from is Observablehq, which has the additional property of being a code IDE which simplifies greatly how you actually upload code to a public location.
The core building block: https://observablehq.com/@endpointservices/serverless-cells
Some blogspam: https://medium.com/nerd-for-tech/towards-a-better-serverless...
I built the IndiaAuth provider using this transparent technique, so you can see exactly how it works.
sidenote: the programming model of Observablehq is a data flow graph and its bloody excellent for hot developing a running program. You can swap one dataflow node for a completely different piece of code and leave the rest of the program state in place. This is finer grain iteration than a REPL.
The topic is reconciliation on top of that... and those kinda of experiments are possible. Observablehq exposes the "this" value to access the previous value of the node. So you can do traditional reconciliation: https://observablehq.com/@tomlarkworthy/reconcile-nanomorph
An individual dataflow at the top level can be seen as a stream of values.
I have not thought much about general reconciliation, but if I were shooting for maximum efficiency I would be thinking about maybe you could add delta sync to these streams.
I definitely observe the situation where I want a dataflow node to be an array of values, or an aggregate result over an array, and then I want to add a single datum to the dataset upstream. Currently everything just gets recomputed from scratch which seems wasteful. I am not sure if generalising reconciliation helps with the issue though.
it's amazing having seamless code sharing between front end and backend, with inline code documentation.
I made a whole oauth server like this! https://observablehq.com/@endpointservices/auth
[1] https://extension.uga.edu/publications/detail.html?number=B1...
Code changes are local operations, so your running program state is generally preserved . This is an amazing boost for iterative development. This is Smalltalk level productivity boosts
Sucks if you want to do something legacy but I want to move forward with programming and breaking the sequential execution model is the right step IMHO.
https://speakingofresearch.com/2015/12/10/importance-of-anim...
Do we want to give up learning the brain? (And other organs but brain is extra special)
It's got: offline first, latency compensation, pubsub, partial sync, authorization rules and serverside timestamps, and global transactions, and 5 9s availability backed by spanner for umpteen languages. https://tomlarkworthy.endpointservices.net/blogs/firestores-...
Having worked in SF and Berlin... SF was definitely better paid overall and I made enough money to buy a nice apartment in Berlin, something which seems unachievable by working in Berlin all your life.
Maintaining a complex Dockerfile is just more crap that makes programming tedious.
As I said, I write good docs, but no-one will read them until they have been caught out 3 times asking obvious questions had they gone to the doc first. If I am not around, I know the docs will be abandoned immediately despite it representing a tome of hard won knowledge that saves an incredible amount of aggregate time from never needing to figure out how to do the same task twice.
https://observablehq.com/@tomlarkworthy/phiresky-sqlite-quer...
https://observablehq.com/@endpointservices/auth
I use this Auth as replacement for Firebase Auth... I can issues tokens for backend services using Indieweb identities that engage with Firebase security rules.
Anyway, I am super excited to have observable/Firebase/Indieweb all interoperable