On the future of Akka and Lightbend

I really enjoyed Play V1. I got some stuff done with V2, but (and I say this as somebody who really liked Scala at the time) I think it absolutely disappeared into the morass and never came back.

Ninja is pretty cool. It works well with Kotlin, which is what I reach for today with the JVM, but tbh the easy path today is pretty much Spring Boot so I have some trouble rationalizing using anything else.

I can concur. Many years ago, I had used Play v1 to solo develop a reasonably functioning website, in part time. But then came Play v2, and for some reason it never clicked for me.

Haven't looked at Ninja. Currently I'm looking at Phoenix as my go-to framework for side projects.

You could say that again, play v1 was so simple and productive. Then v2 was pure insanity. I switched from v2 to golang and it felt like I had been punishing myself.

This is what really sold me in Go as a language and I’ve used it liberally since then. Nothing beats just being able to get your head around things.

In the past I have designed, implemented and brought into production many Akka based systems with medior and senior teams. Most of these were akka-streams based.

The Akka actor model requires a different mode of thinking which can cause problems for experienced engineers. Ideas such as 'let it crash' and at-most-once-delivery are not trivial. Basically systems design based on resilience instead of robustness.

When Actor based systems are well designed, they can be extremely powerful, scalable, resilient and adaptable.

Small nitpick. I found that akka-streams missed many useful primitives. For our toolkit, I designed flows with retry-logic, flows with cache lines and flows with metric measurements (sending data to Datadog).

I've had good success with akka at multiple companies. At Protenus the core ETL is based on akka streams. Which has worked great. No notable issues. Tho we don't run in clustered mode.

At glngn we didn't use akka streams (outside of akka http) but did use event sourced, persistent entities powered by akka typed in clustered mode. Deployed to k8s. Which definitely took effort to set up nicely and enable smooth deployments. Our clusters were not huge so many problems did not show up (split brain).

I haven't used other systems that provided an equivalent to akka typed persistent entities. I can't compare akka in that sense. However the event sourced persistent entities model is really, really effective. Definitely a different paradigm: some stuff is trivial that would otherwise be a trial.

I suspect many of the benefits could be achieved by using something like Kafka feeding non clustered nodes. But never tried.

I've now worked for about six months on a Akka.NET codebase. I find it a very elegant high-performance framework that gives you a lot of flexibility in how you want to solve a wide range problems.

This being said, I've been burned already several times by the complexity and its raw power. The codebase tends to become verbose and difficult to navigate (everything being an ActorRef). Debugging is difficult and coding is challenging for junior engineers, as you said. I found it very unforgiving to mistakes, and it's easy to shoot yourself in the foot if you use the abstractions without knowing very well what's going on under the hood. Edge cases can be very tricky to manage.

I'm still very conflicted about it. One one side the services powered by Akka are quite stable and performant, but I'm not sure the complexity justifies the means.

No. In fact the Play Framework 1.x (before it went to Akka) was simple perfection. Like Ruby on Rails but with so many improvements. Super simple but fast, performant and not really lacking anything a web framework needs (coordinating background periodic jobs between servers required setting a config file but it wasn't a big deal).

In play framework 1 a new API method could be the following

>@Check("administrator") // Authentication scopes allowed for this method

>public static String returnHeading(int id) {

> return createQuery("select heading from Article where id=%", id).getResult();

>}

See how little boilerplate i have? A junior dev can work with this without shooting their own foot. I used to have clients come into the office and ask for a new API method and I'd type out the API in Play Framework 1.x, type the tests (the framework had a great foundation for unit and integration tests) check that the SQL was sane and deploy in half a day. There's nothing more i want. It scaled well horizontally too. You just had haproxy/nginx in front of multiple servers.

Now you might say "But what if you had a web request that took days and needed all cores on all servers to communicate with each other as they concurrently worked on this mammoth task" to which I'd say shut the fuck up.

Play Framework 2.x is when the Play framework adopted Akka. I'm going to be mean and takes Lightbends hello world using Akka as a taste for those unaware of what this beast is: https://youtu.be/rIFqJxMJ1MM?t=428

I'm just going to copy paste some stuff from the Akka wiki to rub it in.

Akka supports multiple programming models for concurrency, but it emphasizes actor-based concurrency and the eventsourced library provides event-driven architecture (see also domain-driven design) support for Akka actors. Akka has a modular structure, with a core module providing actors. Other modules are available to add features such as network distribution of actors, cluster support, Command and Event Sourcing, integration with various third-party systems (e.g. Apache Camel, ZeroMQ), and even support for other concurrency models such as Futures and Agents blah blah blah blah blah.

Sorry I'm being rude but surely even the people making Akka can see the issues? There might be a use case for Akka. But a webserver for a startup is not a use case. Play Framework was built as a web framework and Play Framework 2.x seemed to focus in on a limitation that practically no one would encounter and make everything about that one use case. Play Framework 2.x overcomplicated the entire stack and gave no benefit. There's a reason Play Framework 1.x is being maintained and patched still. The complication from 2.x and Akka is not worth it at all.

i’ve seen at least two cloud services using akka for a component that required long-lived stateful connection management. while i think both ended up being successful because of akka, it required retrofitting a lot of libraries that already existed to fit the actor paradigm, as well as requiring at least one person who really understood akka.

id also say some decisions around akka monitoring and how that has related to lightbends monetization makes it more difficult than it should be to build observability around the internals of akka, ex. emitting metrics on actor queue depth.

My team had migrated about 35 codebases from a mix of Play, Scalatra, Ruby, and Python, to Akka HTTP. The unification definitely made it easier for juniors to start work on unfamiliar codebases, but the same could be said if we unified on something else. That being said, a portion of the services did make use of Akka streams & actors and we’ve found a lot of success there that otherwise would’ve been complex code with locks or synchronized blocks. Stuff that I’d rather not have to onboard engineers with or review their PRs and miss an unlock(). It’s probably a steeper learning curve by teaching them about actors and futures, but it means they’ll be writing safer code by design, and reviewing their PRs becomes easier for me.

Just make sure the actors are small and have one focus. When they get too large and do too much is when they become unwieldy and difficult to maintain.

I found that akka (akka-core) was not just bad, but bad enough that it dragged down the reputation of Scala. Most (maybe all) of the time simple future-based code accomplishes the same thing in a much clearer way, and without sacrificing the type safety that's one of the biggest strengths of Scala.

(Some higher-level things under the akka umbrella e.g. akka-http (formerly spray-http) and akka-streams might be useful; not coincidentally they also tend to be more type-safe)

Same story with Lightbends Play Framework. Using it from Java is abysmal.

Same story with .NET - less and less reasons for F# to exist with each new version of C#. Only F# doesn't even bring any new useful frameworks to the table.

Lesson number one for all adopters of guest languages on other runtimes, the platform always makes them irrelevant with enough time.

And if the platform dies, it isn't only the main language that dies, everyone else that was seating as guest dies as well.