Author here. Happy to answer any questions, vague or intricate! I love bitmap indexes and can talk about them all day.
162 karma · joined June 23, 2021
sam@swlock.co.uk
The wonder if the limiting factor would be _which_ models can actually be compiled into a reasonably sized WASM module (I'm not familiar with this right now--are you aware of efforts in this space?). If there are genuinely effective WASM models that fit into a reasonable sized modules, then it would fit nicely.
All this with the previously acknowledged limitation that it's not yet on mobile (but perhaps a number of edge Pollen nodes could act as ingresses into the cluster in the interim).
I'm super interested to hear how you might employ it though, if you did start experimenting. I'd be interested to learn where it's useful and where it falls short. Please feel free to hit me up on Github or by email (in my profile)!
Good suggestion on listing capabilities, will add a note.
[1]https://github.com/wazero/wazero/blob/2bbd517b7633bf6a126305...
> And what's the public API/stdlib/bindings inside the WASM workers?
Wazero (via Extism) carries the load here. As it stands, the runtime lifts three basic host functions into guest code which enable the RPC-like behaviour, injection of caller-context and basic logging[1], which are in turn referenced in the guest code like in the example[2].
In the reverse direction, guest code exposes it's public APIs via build directives[3], which are handled by the runtime code[4].
Figured that concrete examples might be more helpful here (I hope the formatting works).
[1] https://github.com/Sambigeara/pollen/blob/567e85d5f1407932dd... [2] https://github.com/Sambigeara/pollen/blob/567e85d5f1407932dd... [3] https://github.com/Sambigeara/pollen/blob/567e85d5f1407932dd... [4] https://github.com/Sambigeara/pollen/blob/567e85d5f1407932dd...
I'd agree that Pollen's current cap-enforcement story is limited, I'm not sure what direction I'll be heading in for that, but I was erring on the side of "bring your own enforcement" as a design pattern (ultimately, people deploy their own decision engines as first class seeds in the cluster). Naturally, the enforcement is weaker than the (fascinating) pattern you've landed on--seriously cool.
> and there’s a tiny Clojure-inspired Lisp (“Glia”) that doubles as an LLM-facing or human-facing shell.
This is a _lovely_ abstraction. How does it work? Does the LLM emit Glia directly or is there a translation layer between natural language and the interpreter..?
> It's a les polished compared to what Sam has shipped, but moving fast, and this post has jolted me into sharing a bit before I had planned!
I'm _far_ from polished. I suspect you're underselling your own position here, looks like you have something very compelling. And apologies for the jolt! Certainly happy to compare notes--I (think) I've added my email to my profile.
You could also establish a dev cluster (/environment) where all devs run a local instance. You can iterate on services quickly, expose ngrok-like capabilities by exposing a local dev instance of a server with `pln serve 8080 test_server` for your colleagues to consume with `pln connect test_server`, etc, etc.
A more whacky idea I've not been able to get out of my head which might become possible as the access story solidifies: imagine a customer could access a controlled subset of your companies offering by having a delegated node, running in their own infrastructure, that ultimately you can delegate and revoke at any given time.
ANY decision around network topography or workload placement is a deterministic calculation run by all nodes individually. If all nodes see the same sub-set of peers representing their entire "cluster", they'll all naturally converge on the same view of what the cluster should look like. If the calculated output determines that Node A should claim Seed B, and it doesn't have it, it requests it from a peer who has it.
As soon as the partition recovers, nodes see the additional nodes re-enter the candidate set, which is then added in to future routing and placement decisions.
The main tradeoff to understand here is that you're at mercy of the random (best attempt redundant) placement of a seed. If the entire cluster has, say, 2 replicas stored on any given nodes, if a resultant partition doesn't happen to have either of those two nodes, then the seed will be unavailable until the partition recovers. You can work around this with "smart" initial placements (one near, one close, for example) but you're still at the mercy of random partition events. An additional factor is of course getting very unlucky with dropped gossip events, which would also impact the rate of convergence across the cluster.
I do absolutely despise the complexity of administering modern distributed systems, hence my attempt to make Pollen as ergonomic and (as much as I hate to use this term) batteries-included as possible.
I've not come across either of those projects, oddly. I have a tendency to avoid looking for similar projects during the development of my own, lest I get despondent and run out of steam. Both sound cool, though. I'd say WASM was a natural workload "type" that fit nicely into what I was trying to achieve with Pollen, rather than a driving factor, if you know what I mean.
My intention is for Pollen to become a "generic blob of computational capability" into which you idly `pln seed` a workload and do not have to worry about ANY aspects of managing locality, scale, redundancy etc. You seed a workload onto any node, and you call it from any (other?) node. If you want to add more computational power to the cluster, you fire up Pollen on another machine and `pln invite` -> `pln join`.
Every node also has it's own ed25519 cert. The root key pair (the "don't lose this or you're in trouble" key pair) is used to delegate admin certs to other nodes. I'm also working on a mechanism which allows you to bake any arbitrary properties into a cert (as it stands, these are lifted into the WASM guest code for, say, in-application authz purposes). I have more ideas about how this can be extended in the future.
The root authority can invalidate a participating peer's cert at any point, currently just via a `pln deny` command which is eagerly gossiped around the cluster so other nodes stop talking to the denied node, too. I think this offers some opportunities for some fairly novel applications. Perhaps, in the future, you'll provision a node with a certain level or capability or authority to run on some external infrastructure. It'll have all of the (allowed) capabilities of your cluster, but will act like it's local to the external system. Plus, you can revoke it's access or re-set it's capabilities at any point; `pln grant` eagerly applies across the cluster, too.
The workloads, at the moment, are just anything you can compile to WASM via the Extism PDK. Stateless, for now, but with a view to add shared state and persistence in the near future!
Sorry this was rambly, hopefully it offered something useful.
I'd say "WASM-powered serverless functions" is a reasonable analogy, if your serverless functions maintained a minimal number of live replicas at any one point Also, of course, you're tied to the physical ceiling of the explicit hosts that are underpinning your cluster (N machines which are not dynamic like, say, lambdas are when they auto-provision to match demand).
And yeah, you can also `pln serve` arbitrary services which are then exposed to the cluster, but it's worth mentioning that these will of course not benefit from the inherent, organic autoscaling and locality mechanisms that come with the WASM blobs. I only added it in as a feature so I could retire my (basic) Tailscale usage.
Also, you can `pln seed` arbitrary blobs which can be `pln fetch`ed from other nodes. You can also `pln seed ./public my-site` a static webpage which you can reach from any node with `curl -H "Host: my-site" http://<node-addr>:8080/` (8080 being a configurable port).
My original intention was that the WASM seeds would be the primary workload entity in the system, because it fits nicely with the whole local-first, self-balancing ethos (and WASM modules are blobs that can be gossiped around readily as nodes claim them). That said, you can also register generic TCP/UDP servers on a host (`pln seed 8090 some_service`), which are callable from nodes and seeds (via `pln://service/`).
On statelessness, as I've alluded to elsewhere in the thread, I'm looking at how (convergent) state can be introduced into the system and exposed to seeds, so this would ultimately add another layer of capabilities to the WASM functionality.
On performance: again, it'll depend. I noted elsewhere in the thread the distributed implications of chaining multiple seeds in a call flow. You're not _just_ dealing with CPU, WASM boundary-hopping or even traditional IO ceilings, there's also a component of synchronising gates in proxying nodes (as seed A calling seed B needs to reserve the WASM instance locally until seed B responds, which has knock on memory implications, etc). Its a fun problem, no doubt the story will get better in the coming months. For a local, simple invocation, depending on the nature of your workload, I would expect the standard WASM overheads to apply (there's only a thin layer between the API and the underlying Wazero runtime). For a lot of applications, this should be neglible.
Also a note on placement, if you run, say, 10 nodes globally, `pln seed` will, by default, only place two replicas into the cluster. However, as load is introduced, the seeds will propogate towards it, so a node that's acting as an ingress for `pln call` will generally claim the workload to benefit locality/latency.
The real challenge is gating and reserving “slots” for downstream calls. If seed A on one node calls seed B on another, as it stands, Pollen holds that seed A instance up and waiting (with the memory overhead etc) until the response finds its way back across the cluster.
You can probably imagine how latencies then start impacting this (espesh when a node in USW is generating traffic that needs to ultimately land on my laptop in the UK), not to mention all of the contention from other nodes elsewhere generating load too.
In the demo, I see about 2500rps land on my laptop with 4k-5k generated across 4-5 nodes globally, but this is a multi hop scenario. If a call is only invoking a single, light WASM function, I see much higher throughout.
The project is in its infancy, no doubt I’ll have lots of fun figuring out how to optimise as it progresses!
In the first scenario above, memory seems to be the ceiling, in the latter, CPU.
(Edit: these numbers are really quite meaningless but I wanted to give something tangible)
I use it in place of Tailscale for some homelab applications. I’ve started to deploy other experiments on a “prod” cluster. The demo I showed shows how Pollen responds to a multi-step pipeline type application; two WASM seeds and a single egress communicating over the provided RPC mechanism (`pln://seed…` etc) whilst handling routing, back pressure and the like.
Right now, the workloads need to be stateless. I’m coming up with a story for state at the moment, which’ll likely start as some WAL-like convergent structure with thin (KV store etc) abstractions layered over it. Probably not dissimilar from the pattern underpinning the current CRDT gossip state.