RX has some interesting ideas, but from a practical standpoint, at companies not netflix, it just doesn't work
I shared with the team and people complained about the complexity… so I ended up ripping it out. What people wanted was a test user creation service that worked well and could be maintained… not one that was the most efficient.
I was using RxJava initially because it was being pushed by the platform team and coupled pretty well with the GRPC interfaces we used internally. However, the rest of my team couldn't/didn't want to wrap their minds around the concurrently functional tasking that it brought (which was understandable).
I didn't want the code to have a bus factor of 1 (me).
This is essentially the same tension between static and dynamic typing.
No matter what, all bug free code is complex. Correct code can be written in any paradigm.
But productful code ships with bugs.
Does test user creation need to be bug free? I don’t know. It feels like something where productful test user creation can ship with bugs, even if in reality your test code harnesses should not be buggy.
You can step even further back and ask: when you have essentially unlimited resources, why ship buggy code at all? Netflix has unlimited capital, and it isn’t in a hurry anywhere. Engineers should never choose productful over bug-free. You learn nothing (Netflix’s product problem will never resemble your startup’s product problem) and you forfeit the opportunity to learn (ReactiveX is a good, expressive paradigm).
This is a startup forum, is this true for startups? IMO, yes. Your startup is going to fail. You will not learn valuable product lessons from the journey. I mean people certainly pretend and say they do, but even in the situation of success, Brian Chesky still hasn’t become Walt Disney last I checked. But once you have enough money to be employed for a year, as an engineer, you know, fucking learn how to do something the right way, bug free and performantly.
So I agree your coworkers are fundamentally making an error.
I disagree with this. It mentally changes the thought processes when debugging, which was the biggest issue that was brought up.
> Does test user creation need to be bug free? I don’t know. It feels like something where productful test user creation can ship with bugs, even if in reality your test code harnesses should not be buggy.
It should be bug free at Netflix scale. My service created more customers daily than Netflix signed up monthly. It also sat in the middle of Growth. People use it to gate deployments. It needs to work flawlessly (or at least consistently).
Are there any talks or blog posts on how y’all wired this up, lessons learned etc?
Sorry no talks.
Promises are all over the place too. I understand why someone thought this was a good idea ("I'm doing the scalable thing!"), but in the end it appears completely unnecessary (e.g. doing a blocking get call in all uses).
The problem with Java isn't Java. The problem with Java is that it lets so many people hold it wrong, and the Java ecosystem encourages it.
I've been coding in C recently and really enjoy the limitations. Usually there's only one way to do something.
In the second codebase, things are a bit simpler. The architecture allows for more readable code and far fewer cases where multiple streams might introduce loops or race conditions due to being in separate files and initialised at different times. We felt the benefits of a v2 build ;-)
I’d say RxJS is a great tool to help respond to multiple concurrent data streams (e.g. user interactions happening at the same time as video playback). If you’re not dealing with a lot of realtime behaviour, you probably don’t need it.
In my opinion the complexity ramps up massively if some consumers want a reactive streams pushing data to them and other consumers prefer to request a current static value of the same data. Watch for this and experiment to find comfortable solutions or acceptable compromises.
If you think you need RxJS, take a look at Redux + Sagas and consider a choice between the two. I prefer the rigour and dev tools for Redux + Sagas over the freedom to be hairy with Rx. But observable streams are a lovely abstraction.
But you're right, the same things can be accomplished in other ways with less up front effort. But where it really shines is when you hit massive scale. Having RX and its concepts in place ahead of growth makes the growth easier.
I'd say a decent chunk of engineers just were not competent enough to get it in the first place and then there were those who just refused to put in the time to learn.
i.e. there are many engineers who I truly believe do not even want to put in the effort to be competent in the first place. That or false competence takes precedent as well.