One major difference - ZeroMQ and Erlang. | Hacker News Reader

One major difference - ZeroMQ and Erlang.

jlouisramblings.blogspot.in

54 pointssaurabh19 comments

Contrary to what the author says, ZeroMQ is actually a very good and powerful tool for building dynamic topologies - no idea why he decided that it biases your design towards a "static" network.

I don't know what exactly do you mean by "dynamic topology" here, but I'll try to explain what the author means, from my Erlang programmer point of view. In ZeroMQ you define a "network topology", however "dynamic" it might be. You define "formal" channels for communication. The computing entities use these channels to talk to each other. In Erlang it's just the opposite. The computing entities are the communication channels. Imagine that you have entity A, communicating to entity C, via channel B. In Erlang you can have (and that's the usual way of programming) that channel B to be a process with computing capabilities of its own. So in fact in Erlang the "network topology" is not merely a means of communication between processes. In Erlang, usually the "network topology" is the program itself.

I don't know any Erlang, but, from what I understood (please correct me if I'm wrong), you can consider processes (let's say these are greenlets), connected directly to one another, so all you need to do to send a message is specify whom you want the message to go to. If you want to do, for example, fan-out, you have a process sit inbetween the other processes and distribute the messages.

In contrast, with ZeroMQ you have to have/set up channels, because these are sockets. At least, that's what's apparent to me that the author is saying. Is that correct?

That's correct. In addition, you use the processes themselves to encapsulate logic. So the communication channel AND the logical parts that works with them are essentially the same.

An erlang process has a global PID, a ZeroMQ socket has an address - same difference. If you want, you can run everything over named pipes, UDP, TCP, or via an in-memory transport with ZMQ and achieve the same semantics. Case in point: there are plenty actor libraries built on top of ZMQ.

You can run ZMQ devices which can act as routers, and you can easily create fan outs, fan ins, pipelines, and more.

For a mildly interesting example of building a "dynamic" typology, check this experiment: https://github.com/igrigorik/zeroconf-router

Well, of course, you are correct. The author said that in the article. As long as a language is Turing complete, you can use it to do everything that is possible with the other Turing complete languages. But he is not discussing what is possible. You say that a 0MQ socket has an address. That is one of the points the author is making. In 0MQ you send a message to a channel. In a sense, you might not care what entity is going to receive that message, what entity is going to do whatever computation is required and what entity is going to answer your request (or in what language is that entity written). This is very different from the way we program in Erlang. In Erlang every computing entity is directly accessible (as network and in-memory channels are transparent to the Erlang programmer). Usually you talk to entities (and spawn entities to talk to other entities when you need asynchronous requests), not "through channels", and that makes a really big difference to the way we build our programs. Of course, you might argue that it's all the same, but to us, Erlang programmers, it's not. It's a whole new way of thinking.

From my experience, using libraries on top of ZMQ is not the same as using Erlang directly. Yes, you get the same semantics but it never quite feels right, and sometimes will end up reinventing the wheel to end up writing the glue code that resembles a half baked version of Erlang/OTP.

Yes. It's a different paradigm. ZeroMQ and Crossroads I/O deliberately separate the channel from the computation. That allows packages of messaging logic (so called messaging patterns) to be pre-implemented and available out of the box. These deal with complex and hard to get right issues like error handling, pushback and fairness. In Erlang you have to implement these by hand.

ZeroMQ has been described by others as a socket-scripting language.

You can do Erlang/OTP-style processing over ZeroMQ, it simply doesn't enforce it. In fact, if you wish to do so, you should use (or build) a library around ZeroMQ as an abstraction. The use of that library will resemble the use of Erlang/OTP and will build-in these developer assumptions.

A really good framework built around ZeroMQ is Storm, which does exactly that: https://github.com/nathanmarz/storm

For more context, I'm the primary author of the OpenStack ZeroMQ-RPC driver (which does distributed RPC).

https://github.com/cloudscaling/nova-mq

As long as you have the "over ZeroMQ" part, it's not Erlang style :). The author tries to explain exactly that. In Erlang the "over X" part is omitted. In Erlang the "over" entity is a processing unit of its own. In abstract terms, in Erlang you don't talk "over" something. You talk "to" things. And even better - you spawn things (processes representing requests) to talk to other things, wait for a response, process that response and even make decisions of their own. And that's exactly the point that the author is making. He is not telling what is possible or what is not. He explains the mindset difference between Erlang programmers and such used to the concepts of "socket", "message queue" or whatever one calls them.

I think the assertion is that you can build the same type of abstraction over ZeroMQ that the Erlang runtime provides, at which point the "over ZeroMQ" part is just an implementation detail.

That is correct. But then, after you implement the system in question, you might not want to call that ZeroMQ any more. In the same way that we don't mention ssh when we talk about our Erlang clusters. And I think, you will lose quite a lot by cutting out all the goodies that ZeroMQ provides. You might not want to do that.

Tangential comment here; it's a bit sad now that blogspot redirects the * .blogspot.com URLs to use the country-code version based on the reader's presumed geolocation. Normally a "* .in" link would imply that the author of the article was from India or wants to be associated with India, but for blogspot links in HN it means that the submitter read the article from India or wants to associate the article with India. The difference causes some confusion in me.

> It would be fun to see a language where process Id's - that is names of processes - have a type of what that process receives.

I like this idea. It would allow you to catch and log incorrect messages at the sender. And it would reduce the amount of static configuration by facilitating service discovery. But the type specification would be difficult to design well.

They are adding typed channels to Haskell:

research.microsoft.com/en-us/um/people/simonpj/papers/parallel/remote.pdf

On the contrary, I find ZeroMQ provides a great foundation for an Erlang-like distributed actor system:

https://github.com/celluloid/dcell

Apples and oranges. ZeroMQ is a library. Erlang is a language+runtime.

Reply on news.ycombinator.com