Comment by MCRed | Hacker News Reader

Comment by MCRed | Hacker News Reader

Parent Full thread

Nothing against react or reactive, but it's yet another reaction to going down the wrong path originally -- it seems.

I really don't understand why so many people jump to event driven systems when genuinely parallel computing is so easy. Specifically Erlang/elixir make it easy. EASY I tell you!

Yeah, yeah, Erlang has a weird syntax. Ok, so lose two weeks learning it. But you don't even need to do that-- just learn Elixir which is pretty comprehensible right off the bat.

Where's the downside? There isn't one. Yet people are going out of their way, doing all these different projects and attempts whose sole purpose is to try and fake your way around a problem that was genuinely solved 20 years ago!

To be honest- and maybe I'm wrong and missing something here-- it seems like so many programmers are following what's hip and popular. V8 was a neat javascript engine, and then node cam along, so lets jump into callback hell! Why put up with that when erlang was around and well known?

It's like people choose technologies based on fashion. But that seems too... pessimistic so I don't want to believe that.

I spent a total of 11 months consulting with a company that built a large (50kloc) financial system in erlang. They had terrible performance problems that were caused entirely by erlang.

Imagine you have a large amount of data (order books, accounts, etc). You could put it all in on erlang process, but the gc does not cope well with large heaps (multi-second pauses). You could store the data outside the heap (eg ets) but then you pay the cost of copying on every access and have to tradeoff ease of use (more data per key) vs performance (less data per key). You could split the data up into many processes and then all your simple calculations become asynchronous protocols. Have fun debugging the math or rolling back changes on errors.

I went into that contract with a fondness for erlang. Now I wouldn't touch it ever again. A naive single-threaded blocking server achieved 10x less code, 40x better throughput and 100x better latency. I used clojure, but any sane platform would have worked just as well with that design.

I'm curious whether you brought this problem up to the attention of erlang maintainers, what where their reactions and whether it will be addressed in the future. GCing large objects on the heap was (perhaps still is despite improvements) a significant problem also on .NET platform. Not sure whether this is the case, but have you considered patching the GC yourself or perhaps moving the problem to BIFs and managing that memory manually (without copying to ETS) ?

The problem was not a few large objects, but large numbers of small objects eg thousands of orders per market. I considered using manual memory management but it would have required rewriting most of the code (it was almost all business logic). I sketched out a prototype in clojure to see whether a better GC would help and found that even with hastily written code and much abuse of persistent data structures, the G1 collector could keep the pauses under 100ms. I expect that carefully tuned code would reduce those much further.

50kloc is considered "large financial system" ? Just curious.

It depends. I worked on a trading platform that >1M LOC. The overall complexity was lower than another quantitative modelling engine that only had couple of tens of thousands LOC. Complexity, work and LOC are correlated, but the correlation is not 1.

In the case of the former system, 95% of the code was connectivity, GUI, persistence, etc., the actual business logic was maybe 5%. And I'm not even talking about stuff written in kdb/q, where I produced maybe 500 LOC in a month. :)

As someone familiar (and generally quite happy with) reactive programming, what does Erlang/Elixir do that makes it a more general, better solution? And to what types of problems, specifically?

I think the reason that I picked up Rx is that it was a natural extension of LINQ, both literally and figuratively. What made Rx really click for me was thinking of it in terms of IEnumerable that explicitly acknowledged of the time domain. Since then I've come to understand some of the concepts that underlay both of them (monads, mostly), but I wouldn't have made the leap to Rx if it wasn't so easy to get there from existing knowledge. I haven't encountered anything where the next step would be "learn Erlang." You can make the argument that we should all learn about things that are unfamiliar, but there are lot of things that are unfamiliar and I haven't seen a lot of reasons why Erlang should rise to the top.

There is a confusion of terms here. Rx is reactive programming (aka dataflow). What this article talks about -- isn't (and indeed is better served by Erlang or other similar solutions).

I followed the Coursera course linked from the article and the similar "Programming Paradigms" course from Standford[1], and from what I recall, dataflow and actors where formally equivalent - i.e. you could write any one using the primitives of the other.

Dataflows can be implemented through message calls to independent actors, but I also saw a stateful actor implemented as a purely functional dataflow in the shape of a sequence of its successive states.

I've seen recently a lot of people from both styles saying "but that's not really what 'reactive' is" at the other, and I think it may be a misunderstanding caused by that equivalence.

[1] https://www.edx.org/course/paradigms-computer-programming-lo...

Erlang has two things that are relevant to this discussion: user level threads (aka green threads), and asynchronous message passing.

User level threads reduce the memory footprint of threads, and possibly the context switch time (depending on implementation) but they are not without drawbacks. Heck, Java 1.1 had green threads but they dropped them for kernel threads in later releases (http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_...).

The main issue is balancing work on multicore machines. With kernel threads the OS will automatically balance them. With green threads you are reliant on the user level scheduler to do this work. It is more difficult to get right without the level of information the OS has. If you want to give the programmer control over scheduling you need to provide some access to kernel level threads.

Other issues include: not all the OS IO primitives have non-blocking equivalents you can use (typically you have to use a pool of kernel threads for these operations; the runtime may hide this.) and you still pay some context switch time with you don't actually have at all in a simple epoll / kqueue based system.

I'm running out of time, so I'll just note that asynchronous message passing is hard to reason about.

Ok, I know a bit about Erlang and think elixir is cool.

And for do a professional work, go for it is the right thing.

But here the problem of "just do erlang, is better" and where the JS folks are doing BETTER is that they try to show how do things, not just say this or that is better.

And look: I read that post, and because I see some functional stuff (map, filters, blah) I get the impression that was a good explain!

I know... a lot of JS code around is a terrible hack, and kinda cool/sad/funny how in JS are re-inventing (and a lot of times, poorly) what is better in X.... but as marketing, they are winning.

So:

Where is something like this that show how build reactive properly? And I mean, build, not just use..

And hopefully in a digestible manner. I read a ton of stuff related to languages (because I wanna do one, for fun) and is like this:

http://fsharpforfunandprofit.com/fppatterns/ivorytower.jpg

So I think some examples in how do this well and why are needed.

P.D: For example, http://fsharpforfunandprofit.com/fppatterns is probably the best advocacy that I have found about practical functional thinking. Is the reason I have trying F#!

Alan Kay argued [https://queue.acm.org/detail.cfm?id=1039523] that programming is a sort of pop culture. I think this just illustrates that point.

As an experienced developer it is getting really grating. Why so many jobs requiring NoSQL "skills" when a relational database will solve 90% of the perceived problems. We have good mature back end frameworks. So unless you are writing a chat application, why is everyone looking to Nodejs?

Easier, or at least easy, fun, less compatibility issues when you live in a pure js and json world on the front and backends, productive, good libraries, good built-in libraries, easy to deploy, productive.

Converting to JSON is pretty easy in almost any language.

And the framework I currently use is productive, has good libraries, has good built in libraries. Has a good security record and is extremely productive and stable. Deployment is one area where it doesn't shine. If that's the best reason for choosing a language I would be using PHP.

And it doesn't involve JavaScript, which is in my opinion a benefit.

Reply on news.ycombinator.com