Webserver Back End with Orleans Actor System, Dotnet Core and Server-Side Redux

Neat series. There was another recent post on using a C# version of Redux as well, at https://spin.atomicobject.com/2017/03/13/adapting-redux-c-sh... . It's nice to see Redux's concepts spreading outside the Javascript world. I've had similar thoughts regarding some of the Python services I work with.

If anyone is interested in learning Redux (or React), I keep a big list of links to high-quality tutorials and articles on React, Redux, and related topics, at https://github.com/markerikson/react-redux-links . Specifically intended to be a great starting point for anyone trying to learn the ecosystem, as well as a solid source of good info on more advanced topics.

I also maintain a catalog of Redux-related addons, utilities, and tools, at https://github.com/markerikson/redux-ecosystem-links .

We've been working with Orleans for a few months to build out the next iteration of our infrastructure. I have nothing but good things to say about it. One of my favourite aspects of developing with Orleans is its single-threaded nature. Because a grain can by default only process one call at a time on a single thread, it's easy to make guarantees about data consistency inside that grain regardless of how many processes are reading/writing to it at once.

The idea of a stateful grain is also really powerful and something we're harnessing extensively: if everything is a grain, all of your data is already cached in the activated grain so there's no need for a separate caching system. Additionally, any updates to the data are immediately available to the "cache" before you send it to the database. So transient data can be cached even if you're not sure you eventually want to persist the data. This allows you to do partial application of data to grains, and then eventually commit the data to storage (in the case where writing is expensive.) This of course is totally possible with more traditional architecture, but its impossibly simple in Orleans and it's hard to go back once you get to use it.

I also want to give a big thanks to the folks who are active in the Orleans Gitter channel. Honestly probably the most helpful open source community I've ever encountered. They're extremely thorough and always willing to help.

This was an interesting read! The stack is out of the ordinary and intriguing. I'd love to see numbers, performance number: request per second and whatnot compared to a traditional architecture to back up the conclusion :

> This architecture is highly efficient, almost infinitely scalable, yet very light on infrastructure.

I'm curious on how scalable such fine-grain grains (pun intented) (i.e. one grain per user?) are. Wouldn't communication across all those grains yield to poor performance?

Another big question I'd be interested in is testing and debugging: Redux tree/reducers model let's you easily test things. How's the testing/debugging experience in orleans? The author mentions the "time travel", I'd be curious to know his experience so far.

> An Actor system makes sure that every data element / actor object lives in only one place (one machine in the cluster), so there is no duplication of data, which simplifies things dramatically.

What happens when that machine goes down? My understanding of Orleans is that it'll be fault tolerant and handle this, but what are the impact on performance? And is there any guarantees the data will be stored without corruption in the external storage (azure table in the author's case)

Very excited to see this. I built a large Actor-based architecture (homebrew originally, now ported to Orleans) and while spinning myself up on React/Redux recently I started realizing how closely our data flow architecture mimics Redux.

My plan had been to work to make the parallels more explicit so it's easier for other devs to wrap their head around the architecture ("it's just Redux" is a lot easier to explain than "it's this custom homebrew thing", just like "it's just Orleans" helps significantly with detailing the Actor model).

The idea of an actual server-side .NET Redux implementation is really exciting, particularly coming out of the Orleans community.

I built a fairly large backend system using event sourcing ideas and am now using Redux for another backend tool I'm building. Redux and event sourcing are very similar. One big advantage of using Redux is there's tons of documentation and blog posts and plugins so you get a lot of power out of the box. E.g. it took me 5 minutes to add in support for the Redux devtools https://github.com/zalmoxisus/remote-redux-devtools.

I created an architecture nearly exactly like this named "Fuzed" quite some time ago: https://github.com/KirinDave/fuzed

Our findings in production were very much inline with this without the additional wrinkle of Redux server-side (which I decline to comment on), and of course Erlang doesn't virtualize Actors.

Sadly the SPOF-mitigation I did was never really released integrated with Fuzed.

But this kind of system offers FANTASTIC flexibility when done correctly, and allows you to do what many AWS-based folks also enjoy doing: allowing new and development resources to run peered to production resources for limited trials and easy experimentation.

I skimmed through the paper and one possible problem is when actors communicate with each others it might create a lot of network traffic, especially if one node goes down, it has to sync and might create a load loop as the load of the other nodes increases. A better way is to let the client do the orchestrating of talking to different nodes, so that no server-to-server communication is needed.

Does it mean it would be possible to use redux with either squeak or pharo?

What Orleans like systems are there in other languages?

Is this the same problem space that meteor solves ? Does anyone have a perspective on what's unique to each ?

Does the throwaway line

> The Redux reducer that processes the action server-side in C# is almost the same as the one in typescript working client-side

... bother anyone else? Surely this is the worst-of-the-worst kind of non-DRY code you can write, where you have to have two identical implementations in two different languages, but they also have to have other responsibilities? Even if you did write tests for both, you would have to make sure that the tests were testing the same things.

Having thought about this a bit since I started typing, I think your server-side redux has to be different in some fundamental ways. It won't be an in-memory representation like the client has. It will have to actually perform the DB operations. The server will care about different state than the client. The server often won't care about in-memory representations, so won't need to produce a new state upon every action. If it does, it might be in a different form - a cache that keeps (f)recent data, for instance, rather than an entire state tree that evolves.

I'd like to hear from someone with experience on this, who has written two reducers for the same dataset, perhaps in different languages. Did you find yourself with subtle re-implementation bugs? Was it actually more productive than providing a regular REST API and having your client-side Redux store persist itself?

I personally have to wrangle lots of highly structured, partly denormalised data, with a quite a few layers of SQL indirection. Moving one part of the nested data to another branch, for example, is tricky to get right, so I test it thoroughly. And then I have to deal with the fact that my requirements include undo and save buttons – there is a constant distinction between previously-persisted and newly-added objects. I could not imagine having to write this kind of code twice and keep the two implementations in sync. But here's a shot:

- The server would act like a side-effect hook à la @ngrx/effects.

- Make a redux reducer tree out of multiple grains. Each one gets dispatched all the different actions received by the root, so you can have both a `Parent` grain and a `Child` grain react to a `ChildMovedToOtherParent` action. This is only necessary for me because of an over-denormalised schema, but I can imagine other uses.

- Handling undos is tricky. On the client, it's easy, because everything's in-memory and immutable; just store previous states. On the server, it's much harder. Event Sourcing would have you just pick a point in time and rebuild from there. I can see that kinda working: instead of writing to the DB immediately, queue up the actions in-memory and execute them when the server hears a Save action, creating a point you can't rewind past.

- All of this sounds a bit unstable, however. When you queue up DB calls, you have to execute them all sequentially, even if it's just modifying the same record over and over again. The most efficient way to do this, without running a hundred useless DB calls on save, would be simply to hold all your objects in memory, using a reducer to compute their appropriate final to-be-persisted state. This is way too complicated.

Has anyone got any experience working with an architecture like this? Or is everybody that's using Redux for state management only as ambitious as a counter or a one-way data funnel and not one bit more?