Synit – A Reactive Operating System | Hacker News Reader

Synit – A Reactive Operating System

synit.org

119 pointsgjvc36 comments

It took me several minutes of clicking to determine that it's an init/process lifetime management/IPC thing for Linux, that uses the "Syndicated Actor Model" for dependencies and IPC.

They've chosen some interesting terminology that's a little different than everyone else.

If it had been around by 2010 (which predates a lot of the research it's based on, so no) when many Linux folks were looking at SMF/SRC/launchd and thinking they wanted something like that, it would be more likely to be relevant, but I think we're going to be stuck with the systemd/dbus stack for some time due to network effects and its all-or-nothing design, whether or not someone comes up with a technically superior solution.

Author, AMA

Hi! I have a few questions:

1) What are the benefits of the reactive operating systems? Do you have any example use-cases that this does better than traditional approaches?

2) Do you find this related to functional reactive programming at all?

3) Since this is a model of concurrency with eventual consistency, do you see it benefitting from eventually-consistent OT or CRDT data types?

I am asking because I am working on what might be a related model: https://braid.org and https://stateb.us. We are building a "distributed state abstraction", that we envision will end up in three places:

a) HTTP will upgrade from a state transfer to a state synchronization protocol

b) Applications will be separated into UIs on top of a "web of state" (see https://stateb.us/static/statebus-demo-3-31.mp4 ) and transition from web apps to app webs

c) Operating Systems will replace file systems with state systems; where local variables in memory can persist to disk without explicit read/write calls, and can be read/written across processes without programming overhead.

So I am asking these questions because I wonder if we are all looking at the same programming abstraction, but from different perspectives!

Thanks for the questions.

1. Applying the same tools used in the SAM for managing concurrency and interactivity "in the small" to "larger" aggregations of components, and starting to get insight into the semantics of operating systems, if you like. For examination of the advantages and drawbacks of the SAM, see https://syndicate-lang.org/tonyg-dissertation/html/.

2. Loosely, but only loosely. My dissertation (linked just above) discusses FRP a little, but in a nutshell, the SAM has ambition that covers more area than FRP.

3. Yes, absolutely. I've done some experimentation, but nothing rigorous yet.

From the points you mention, I think there might indeed be some common underlying thing here we're looking at from different perspectives!

Thank you! Hm, so is it safe to say that this is an experiment for you in applying the SAM to a real operating system, to see how the model works in practice?

Yep. And so far, so good. I think it works well.

In your dissertation you mention that "program" internals such as Exceptions should not be asserted as facts and pollute the data-space; only relevant facts/knowledge about the domain should be shared.

I understand this point, but what if you want to express knowledge on how to adapt the technical parts of a system and its infrastructure, not just the "problem" domain? I imagine that Syndicate and its abstractions could be very helpful in this regard.

What are your thoughts?

Edit: For example, you may want to delegate the diagnosis of critical Exceptions to engineers, perhaps suggest resolutions, patches. Would you use a dedicated syndicate network for this? What would be your approach?

Great questions.

Yes, that's fine: if your domain is implementation detail of the system itself, then using Syndicate to allow different program components to have conversations about that domain is totally OK.

The point I was trying to make about exceptions and polluting the dataspace is as a point about unnecessary coupling: exceptions per se are not in most domains, so communications mechanisms that include exceptions as part of the transport are in some sense wrongly factored. Likewise transports that include sender information at the point of delivery of each received message. The point isn't that sender information or exception details aren't useful, but that they're not something to bake in.

Concretely wrt your example: you could use a dedicated dataspace for such things, yes, or you could design the language of discourse for a large shared dataspace to allow multiple ongoing conversations at once on unrelated or semirelated topics.

Can you clarify briefly, is an error communicated out-of-band? Or is it not communicated at all?

For example, if I send a request for a key that's "not found," I would think a standard out-of-band error is what you mean?

On the other hand, if I send a request to put data in a key and there's a version mismatch (say my schema is newer and I've got an additional field) -- then silently allowing it to proceed makes sense, sort of?

Or did I misunderstand completely? :) I realize this is rehashing things at a fairly beginner level, but it would help clarify a lot!

By "error" I usually mean some kind of crash (exception/termination/disconnection).

Let's imagine that two actors are interacting. Actor number 1 is observing assertions matching pattern (A, *), and actor number 2 then asserts (A, 123).

Actor 1 then receives an event indicating that a tuple (A, 123) has been asserted.

Now, imagine Actor 2 crashes. The Syndicated Actor Model ensures that all its assertions are cleanly retracted. This causes an event for Actor 1 saying that (A, 123) has been retracted again.

So you can see this as a communication of the crash! But what it doesn't do is indicate any detail about why the retraction happened. It could have been a deliberate retraction, or a crash due to any number of reasons, or a timeout in some intermediate transport layer causing disconnection, etc etc.

There are other uses of "error": for example, your "key not found" status. There, the protocol for interaction among actors should include an explicit assertion or message. This is the kind of "error handling" seen in languages like ML, Haskell, Rust, and so on, where errors are (non-crash) values. It's a good strategy.

At a slight tangent, there's one thing Syndicate can do here that other (non-actor) systems cannot, and that's "key not found yet". For that, silence is a tacit indicator of no result. For example, you could assert interest in (Map, "mykey", *) tuples. If and when any such appear or disappear, you get notified. Because time is involved, receiving no message is an indicator of "key not found yet".

Thanks for that! Very helpful.

Now I want to get into the weeds, if you have a minute more. For "key not found yet," is there a way for Actor 1 to know the difference between -

1. Key not found, it's too early to know why. Kind of like a "yield".

2. Key not found, and any Actor has now had plenty of time to supply one, so it's safe to move on (update the UI, etc.). Kind of like a "nil".

There are a couple of approaches here. If there is a moment when it's known that no matches will be found, then either an actor can assert something to this effect or the querier can wait "long enough" and then move on. For the former, if there's some actor in charge of maintaining an authoritative copy of the collection concerned, they may assert some "no match" record. For the latter, the SAM includes a "sync" action which is essentially a no-op roundtrip to some target. Once the sync returns, you know all previous assertions must have been seen by that target, and if you know something about how the target operates, it might be safe to conclude no response will be forthcoming.

I see. Thanks for the clarification!

Another question, Smalltalk related.

What I noticed in various Smalltalk systems is that there is a lot of code duplication for handling UI interactions (such as dragging or selecting objects in various kinds of views). Garnet, for instance, addresses that problem with Interactors and constraints [1].

Also, in Smalltalk, you will find countless instances of re-implemented tree data-structures e.g., package trees, class trees, UI component trees, which increases maintenance effort and makes the system less uniform than it perhaps could be.

Could Syndicate address these problems?

[1] https://dl.acm.org/doi/abs/10.1145/67449.67512

Syndicate certainly lets you factor out some kinds of repeated behavioral patterns: I discussed in my dissertation (chapter 9 [1]) examples around cancellation, around state-machines, around observers, and around demand-matching.

Besides these examples, the general facets-plus-conversational-context idea lets you factor out composable behavioral traits. Which word reminds me of the Traits of Schärli/Nierstrasz/Ducasse [2]: those Traits have proven benefits for improving reuse and reducing redundancy and bugs in Smalltalk systems. So perhaps in general what we're looking for are new perspectives on old problems - new ways of isolating and composing ideas.

Strictly on the example of dragging objects, I actually have a little demo GUI that factors out object draggability into a composable facet of behavior that can be "mixed in" to an actor representing a graphical object. See [3] and, for detail, [4].

[1] https://syndicate-lang.org/tonyg-dissertation/html/#CHAP:PAT... [2] https://dl.acm.org/doi/10.1145/1119479.1119483 [3] https://git.syndicate-lang.org/syndicate-lang/syndicate-gui-... [4] https://git.syndicate-lang.org/syndicate-lang/syndicate-gui-...

Thanks for the link to Interactors, by the way, I will enjoy having a look at that paper!

My pleasure! I found it to be a refreshing read. Thank you very much for your response. I will study your examples and read up the parts in your dissertation.

“Reactive programming” is promising. I’ve not looked at OSs, but for other things, like databases, I’ve come to the conclusion that our typical mixed-paradigm languages themselves are clunky and likely need some changes. There’s a lot of boilerplate and callback indirection and so on to build primitive things.

Did you get this feeling as well? More generally, how invasive is the “mental model” in relation to existing stuff? Can you benefit from it when mixed with “non-reactive systems”, or do you get the “worst of both worlds”?

Good question. Yes, I get that feeling too.

I find that re-presenting existing systems in terms of syndicated state and conversational contexts, like you're using an FFI, makes everything fit together fairly well. Lots of systems offer a reactive interface of some kind, so it's often a matter of just filling in the gaps; even where nothing "reactive" is available, one can always get by with polling for changes! For example, adding a thin reactive layer to Caddy would get you a nice pluggable systemwide HTTP service. Or, as in synit, wrapping eudev and netlink gets you reasonably coherent interfaces to plug-and-play and to ambient network environment changes.

It's not quite embrace-extend-extinguish, it's more encapsulate-embed-embody. You write "drivers" that encapsulate non-SAM things, embed them in larger systems, and over time take on functionality from the encapsulated componentry as appropriate.

Do you have any plans around GUIs? I caught the capabilities bug about 25 years ago and as far as I can tell this is still the big blocker for a capabilities-based desktop OS .

I'm not there yet, no. I think the existing research on ocap GUIs is promising, and I have ideas for UI in general (think: cloud shells) that might stretch to GUI as well.

Why eventually consistent? Id distributed systems can do strong consistency across computers why do we need to compromise within a single computer?

The idea is that we already have good tools for expressing strong consistency within a single failure domain, but language support for reliable programming with multiple failure domains is lacking. Lots of inside-a-single-computer systems are eventually consistent, or mini distributed systems, so good support for these is important too. (Examples: Container lifecycles. DNS. Filesystem operations. DBus. Systemd management operations. inetd. X11. NFS. Pipewire. mDNS.)

I dug around for a little bit. Can you link to something that points to intent rather than the implementation?

Yes! I think the project page for the NLnet-funded part of the project brings across some of the intent: https://syndicate-lang.org/projects/2021/system-layer/

Deeper background would be something like "build something that is to the Syndicated Actor Model as OTP is to Erlang", because I believe the SAM is a great way to build interactive programs.

Thanks for the AMA.

- Any comments/references on the potential applications where the Syndicated Actor Model is going to be a particularly good fit? (I'm curious, for example, about the potential in robotics)

- Are you targeting any of these with Synit, or is it meant to be a proof of concept?

Proof of concept, to be sure. So far. It's currently on the back burner because I have to eat : - )

Re applications, anything interactive and somewhat loosely coupled. Particularly good for internet scale stuff, I suspect, though again I've not rigorously explored this yet. Tightly coupled control isn't a great fit. I suspect some aspects of robotics could work well with it, but not the very realtime parts. I'm not experienced in robotics.

More in the "anything" category, but what do you think of Lisps and Common Lisp in particular? Practical to write an OS in it?

I'm an old Schemer (well, middle-aged) who has gone apostate in favour of Smalltalks : - ) Schemes and Lisps are fine systems and I enjoy working with them. And yes, practical to write an OS in, but (and this won't surprise you at all) probably even better if you add SAM support to them first! I have done so for Racket thus far.

Interesting. If I may ask: What made you switch "sides"? And what do you miss in Smalltalks coming from Scheme?

I miss macros sometimes. The main thing I think Smalltalks get right is uniformity: everything (well, almost everything) is an object. If I remember rightly, R4RS (maybe even R5RS) Scheme has exactly 9 (nine) distinct types of value. Nine? Really? Who ordered that? I think an ideal balance between the two would be something with uniform object-orientation for behavioural interaction with values plus programmable "views" [1] for destruction of values as data. So something with Smalltalk's extreme late-boundness, plus ML-like sums-of-products for data.

[1] https://www.cs.tufts.edu/~nr/cs257/archive/phil-wadler/views...

That's interesting. Thanks for the paper. I skimmed through it, but I did not get every detail, since it's been a long time I worked with languages like ML or Haskell and my formal background is a bit rusty at the moment.

I do like the premise that, if I understood correctly, (1) you want to specify your computations with the most suitable representation(s) and you want them to be inter-operable. Also, (2) I find it interesting how the tension between pattern matching on explicit representations and abstract data types with information hiding can be reconciled.

On a less formal side, the Personal Information Environment done at Xerox PARC, extended Smalltalk with perspectives, which addresses point (1) from a different angle [1], but the multiple perspectives system is bolted on and requires tooling. Also the scope is a bit broader, since PIE also addresses the integration of non-functional artifacts (documentation and so forth).

I don't know if that can be done more cleanly without rethinking the entire object and interaction model. Taken (1) a bit farther, you end up with a configuration language (that Alan Kay talked about in a seminar [2]) and the system is responsible for delivering appropriated objects to the methods (or processes).

For larger systems I consider that as a must, because without that kind of modularity your components accumulate all sorts of different contexts, which makes it difficult to understand the evolution of the system or take requirements out of the system. Extension methods seem more of a work-around than a real solution. Traits are fine but the composition always ends up to be a global fact in your system, even if it is irrelevant (or even ill-suited) for other contexts.

So what we need is a system that can dynamically adapt itself to different situations, in such a way, that the programmer (or user) can adapt the system locally without having to know the entire global configuration -- which does not work if you have conflicting behaviors but you want to keep them for, say, analysis purposes. In the end, we want the system to get us into a state where we can do experiments as fast as possible instead of doing 99% of (volatile!) preparation work all the time.

But this is of course an extremely complicated problem (theoretically and practically even more so with the growth of software) that needs adequate methods and languages. And this is where syndicate might close some gaps, especially since you are applying it to the messy domain of operating systems configuration and the insights we can use from that.

[1] http://www.bitsavers.org/pdf/xerox/parc/techReports/CSL-81-3...

[2] https://www.youtube.com/watch?v=QjJaFG63Hlo&t=4816s

Edit: To make my long winded post a bit clearer, many of it boils down to the illusive context problem. But rethinking the mentioned issues from a conversational point of view seems very promising to me and could address many issues, such as composition and discovery of components (or better yet, behavior), pragmatical concerns such as optimizations, keeping track of requirements -- all put under the umbrella of multi-faceted conversations within the system and through its users.

This thing looks interesting. But I don't get it.

What's exactly novel about all this?

I see an untyped(!) DSL to model actor supervisor hierarchies, including a protocol which allows the actors to access some shared mutable state.

But else? (Maybe besides some NIH and "rewrite the world" allures?)

From a systems architecture point of view this looks actually very good, but all the details are more or less standard stuff found in industry products since quite some time.

Don't get me wrong, the engineering behind this systems architecture is quite impressive!

But it's engineering, not research to me.

The other thing is, all hard problems, there where the "fun" starts, seem actually unsolved. Form the end of the dissertation defense slides:

  Future Work

    Enhancements to the formal models
    - types; names; grouping & relaying; formal properties
  
    System model
    - performance; tool support; generic protocols; process management;
      stdio; contracts; security; persistence; polyglot
  
    Distributed systems
    - unpredictable latency & scheduling; message loss; subnets

I'm not sure this is actually easy to solve. The basic idea here seems to be that actors coordinate with the help of shared mutable state (the "dataspace"). At this point you're actually back to square one as synchronizing shared state is the core problem with distributed, concurrent systems…

Given that, and the fact that this whole thingy is untyped makes me skeptical. Not even the std. ingredients like type-state, session types, behavioral types, or the like seem to get used. I also don't see linear types here, which go very well with session types / behavioral types, and can help making typing message passing easier.

Also that this approach would be completely network transparent, as I see it, seems like a design flaw. One needs control over the various effects that can occur! Otherwise the points form "distributed systems future work" can't be solved, imho.

And form the practical point of view: There is no Akka / Pekko based Scala implementation? I'm quite surprised, to be honest.

Akka / Pekko is the actor runtime, and Scala is famous for it's eDSL features; besides of course its type-system, which makes things like Libretto possible, which seems related here:

https://github.com/TomasMikula/libretto

Syndicate-lang looks OTOH more like the usual callback-hell.

But like I've said, I don't get this whole thingy really. I've just clicked through some parts of the web-sites. Most likely I'm just overlooking the elephant in the room. So would be glad to learn more!

(BTW: Very nice web-site design on all the projects. Especially nice typography. Really enjoyed this part while clicking through this stuff.)

Thanks for the feedback.

> What's exactly novel about all this?

See the dissertation, which I successfully defended :-) Less flippantly, the novelty is in the failure handling and the communications mechanism.

> I see an untyped(!) DSL to model actor supervisor hierarchies, including a protocol which allows the actors to access some shared mutable state.

There are multiple levels of language here: the DSL for describing system services and supervision is untyped, in the same way as systemd, sysv init, Erlang's supervision hierarchies, and many (all?) others are.

The interactions among actors is also untyped. I'll discuss this a bit more below.

The way an actor makes use of the interaction machinery, however, is as typed as the host language itself. The TypeScript and Rust implementations are statically typed at the level of individual actors.

> NIH

Is there something specific you have in mind here? I'd like to hear about it. I think most of the choices I've made that seem NIHish are defensible, but if I'm wrong, I want to know about it, because it's always nice to be able to not implement something and to reuse something else instead.

> From a systems architecture point of view this looks actually very good, but all the details are more or less standard stuff found in industry products since quite some time.

This is not true. It's in the details where the novelty is. (And, the hypothesis goes, that the specific details chosen make it easier to get a good architecture without heavy effort.)

> But it's engineering, not research to me.

Do take a look at the project proposal, https://syndicate-lang.org/projects/2021/system-layer/, which lays out a research question and so forth. I am sympathetic, though, since I have a hard time seeing the dividing line between engineering and research when it comes to systems work myself.

> The other thing is, all hard problems, there where the "fun" starts, seem actually unsolved. Form the end of the dissertation defense slides:

So I defended at the end of 2017. Since then there has been a lot of progress!

    Enhancements to the formal models
    - types; names; grouping & relaying; formal properties

Regarding types, my colleage Sam Caldwell has been exploring various kinds of dataspace-specific type system: https://www.cambridge.org/core/journals/journal-of-functiona...

Regarding grouping and relaying, the Synit project has made a solid contribution to the relaying part, and I'm currently investigating the grouping part further.

Regarding formal properties, nothing yet has been done.

    System model
    - performance; tool support; generic protocols; process management;
      stdio; contracts; security; persistence; polyglot

Performance is now more than 30x better than the research prototypes of 2017. Performance is now "good". (Faster than DBus, for what it's worth ;-) ) Tool support is improving, though slowly, because it's mostly just me working on it. There's TypeScript language support, for example, making programming with Syndicate in TypeScript relatively ergonomic in vscode, emacs etc. Generic protocols, process management, stdio, polyglossy: Synit has made a solid contribution here. Contracts: investigation of gradual behavioural type systems should lead to interesting results here. Security: Synit and improvements to the SAM have introduced object capabilities (with a unique twist for incorporating assertions). Persistence: I'm actively working on this right now.

    Distributed systems
    - unpredictable latency & scheduling; message loss; subnets

Needs more investigation. The relaying that Synit has introduced is a big step toward getting a good handle on this.

> I'm not sure this is actually easy to solve.

Well no, it's genuinely tricky.

> The basic idea here seems to be that actors coordinate with the help of shared mutable state (the "dataspace"). At this point you're actually back to square one as synchronizing shared state is the core problem with distributed, concurrent systems…

No: please go look at the dissertation some more. The key difference between simple mutable state or tuplespaces and the Syndicate idea of dataspaces is the way they handle failure and the kind of eventually-consistent synchronization model they offer. TL;DR is Syndicate's "shared" state is the logical sum of assertions placed in the space by its members, so retractions (or crashes) are handled by re-computing (or incrementally updating) this sum and propagating changes downstream.

> Given that, and the fact that this whole thingy is untyped makes me skeptical. Not even the std. ingredients like type-state, session types, behavioral types, or the like seem to get used. I also don't see linear types here, which go very well with session types / behavioral types, and can help making typing message passing easier.

These are all open research areas. I'm not sure they qualify as "standard" yet. That said, all these approaches are worth investigating in a Syndicated Actor setting. Except perhaps linearity, which doesn't readily apply.

> Also that this approach would be completely network transparent, as I see it, seems like a design flaw. One needs control over the various effects that can occur! Otherwise the points form "distributed systems future work" can't be solved, imho.

I don't understand your point here. There's no network transparency in the traditional sense here; rather the opposite! Local interactions are presented as if they were potentially remote.

> And form the practical point of view: There is no Akka / Pekko based Scala implementation? I'm quite surprised, to be honest.

I'd love to see such a contribution, if you're interested! Akka is a solid system, of course, but its goals and properties are quite far from the kinds of things Syndicate is aiming for. Producing a Syndicate for Scala could be interesting, though.

> Syndicate-lang looks OTOH more like the usual callback-hell.

Heh well ok. Take a look at the way facets structure conversations. It's actually remarkably close to "structured concurrency". https://syndicate-lang.org/tonyg-dissertation/html/#x_2_5_0_...

> (BTW: Very nice web-site design on all the projects. Especially nice typography. Really enjoyed this part while clicking through this stuff.)

Thanks :-)

Seeing this made me look a bit more at postmarketos. That seems like an impressive project. I just wish it were able to run something like an Android compability layer.

I'd love to have a full GNU/Linux phone with root, VNC, and a (mostly) Android userspace. Termux can come pretty close, but there are issues.

Take a look at Waydroid for the Android compat layer.

Reply on news.ycombinator.com