ZeroMQ – Relicense from LGPL3 and exceptions to MPL 2.0
github.com
github.com
ZeroMQ was absolutely solid and stable; incredibly trouble free and the only issues we ran into were when IT teams didn't open the ports we documented in the configuration procedure. (The resultant architecture actually looks a lot like Flink)
In any case, ZeroMQ is a fantastic piece of technology that I feel like I don't see out in the wild quite enough. It's dead simple and incredibly stable from my experience.
I remember we had a similar use-case. This was for collecting live statistics from a sizable Varnish Cache cluster. We wrote a in memory database to store the data. It's been chugging away for about 10 years now and last I heard the zmq traffic alone was about 3Gbps with zero issues.
IIRC it does all I/O on separate background threads which means every operation actually goes through an inter-thread queue. Which can be good and bad.
We even went as far as writing a Golang version of the VCS server to better handle some of the scaling challenges. I don’t remember the exact library that we are using to call into ZeroMQ, but the CGO overhead was minimal.
For us, delivering an on-prem commercial off the shelf solution, it was untenable to expect the customer IT team to operate a separate, relatively huge piece of tech (remember, this is 2014). Maybe the heuristics would be different today with K8s and advancement of Kafka. But ZeroMQ as an in-process, distributed messaging layer is dead simple. If your use case requires anything else on top of that, it's on the team to design the right solution like resiliency, statefulness, etc.
For a high throughput, distributed compute focused use case, I think ZMQ is still a great choice for coordinating execution. But Kafka and other options on the market now are great choices for higher order abstractions.
You just gave me openstack+zeromq flashbacks
What do mature network applications do instead?
> assert(data_buf[4] < 8);
While your protocol might guarantee that data_buf[4] should always be a value less than 8, you don't use assert() to check it because it aborts the program if the check fails. The proper thing to do is a range check that returns an error for a protocol error (malformed data, etc.).
ZeroMQ literally called assert and any bad data coming in over the wire would cause your app to abort. Insane.
Here is an example bug report:
https://github.com/zeromq/libzmq/issues/186
Keep in mind this was a LONG time ago! So this is not an indictment of the current project!
Having used both (from Python) I found NATS much better. ZeroMQ in particular caused a lot of problems around the HWM (High Water Mark) limit for me or around process starting order (who creates what channel). Having a separate server helps with the last problem.
I think once the _in-process_ constraint is lifted -- you can install or rely on an external messaging server -- then the field becomes much wider in terms of solutions to pick from.
BTW, the way we solved for a similar HWM issue is that we decoupled the ingestion of events from the distribution of said events (with ZMQ). So one process was ingesting events and would send it to a coordinator process that would then send events to processing nodes. The coordinator would reply with it's current queue size and the ingest process would do a back-off once it reached a threshold. This allowed us to stop/start any node in the cluster in any order.
We (at work) use ZMQ + our own special sauces to run vehicle automation messaging stuff. Running some enterprise-ish broker in that environment just isn't on the menu.
Sure, to get the most out of NATS you should run it as a server (which frankly isn't difficult as its a single Go binary).
But you can embed it if you so wish. Indeed if you look at the NATS source code[1], that is exactly how NATS do their testing. They use an embedded NATS server in their test routines.
http://wiki.zeromq.org/blog:multithreading-magic
There's nothing wrong with the opinions presented there -- but it really wants to own your whole threading / serving stack, and isn't really compatible with e.g. a Rust async tokio binary where threads can "move around" etc.
However, it's not that hard to work around their non-thread-safe nature with a single dispatch thread (e.g., see <https://github.com/nyfix/OZ/blob/master/src/transport.c#L457>), and that approach has other potential benefits.
In any case, newer socket types (e.g., ZMQ_CLIENT) have since been defined that are thread-safe, but necessarily don't support multi-part messages. (They tend to be different in other ways as well -- e.g., ZMQ_RADIO/ZMQ_DISH are "sort of" replacements for the original ZMQ_PUB/ZMQ_SUB sockets, but have other constraints as well).
Disclaimer: I don't work with any of that tech right now, but I'm looking forward to working with Elixir, which is in the same ecosystem.
But the truth is that ZeroMQ has a steep learning curve, and some rather sharp edges that can leave nasty cuts if one is not careful.
If you're interested in becoming familiar with ZeroMQ, you might want to check out our implementation: <https://github.com/nyfix/OZ/>.
I feel like I've heard of many larger companies doing relicenses on their open source without this kind of effort.
There is more background here: https://github.com/zeromq/libzmq/issues/2376
Sometimes companies who do "FOSS" make you sign some sort of agreement that they own the code you produce and you won't have any say about re-licensing, so maybe the projects you're thinking about have done that?
Linux doesn’t have a CLA, and it’s the most popular operating system in the world.
If you believe in software freedoms, then there will never be any reason to need to relicense, nor would you want to.
Free software is an ideology, like human rights. You can’t use it only sometimes and be said to support it.
When you publish an open-source project, people are going to assume you want your project to be open source. This often provides an enormous boost to the project, as people are way more willing to contribute to a collaborative community project than just donating time to some for-profit company.
I am totally fine with companies making proprietary for-profit software, but don't leech off the open-source community by pretending to be something you are not. I am at a point where I assume any company-backed project with a CLA is going to do a bait-and-switch as soon as that becomes the more profitable option. Remember kids: corporations are not your friend.
The linked article is precisely a counter example to this point!
The Tivo-ization process of the 90s shows that while this might be frequently true, it isn’t without exception. From a practical standpoint, continuing to provide for user freedom would have been best accomplished (personal opinion) if many projects had been able to move to a more AGPL style license.
> For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable.
My amateur understanding is that the major kernel copyright holders are essentially comfortable with Tivoization and aren't looking to rock the boat with a lawsuit.
What if there was an extreme license that simply said you have to share it upon request from anyone, even private versions? Ignoring whether that's annoying or whether it's enforceable, would that be non-free?
I've seen an argument that the particular way the AGPL is worded makes it non-free, which seems pretty plausible, but I don't think that's an argument against "a more AGPL style license".
Are you claiming the MPL is not a free software license?
Placing your contribution in the public domain is highly unlikely to be possible as your contribution is in fact a derivative work.
Patches are obviously a derivative work. No one spontaneously describes deleting several lines of code & then replacing it with other lines of code.
Ironically, despite all the (unequivocally 100% wrong) yammering about this topic on places like this forum, many of the bigger "evil" companies like Meta and Google don't require transfer of copyright to contribute to their FOSS projects, while places like the FSF do require it so they can relicense under potential future FSF licenses e.g. a practically stronger version of the GPL 3's "or later versions" clause. And there are even more agreements like the FSFe's FSA that can stipulate exactly a fixed set of licenses that might be used in the future, as a sort of middleground.
[0] https://heathermeeker.com/2021/06/01/fsf-drops-assignment-re...
A CTA (transfer vs license) does allow unilateral license changes after the fact.
You can do license changes without that however as long as:
1. You license per file in the repository. This can be quite arduous but many projects will move the old licensed stuff into a sub project to make that easier to grok.
2. Your new license is compatible with the old license.
https://github.com/zeromq/libzmq/issues/4315 and https://github.com/zeromq/libzmq/issues/4455
I wonder what made the maintainer change his mind.
When you do your own license though lawyers need to figure everything out for just your project.
MPL works on the “source file” level. You have to release any changes you make to MPL licensed files, but you can link those files into a closed source program any way you like.
You can static link code under the equivalent version of GPL license. The point of the LGPL was to compromise so non-free software could still use free libraries. I was unaware of the static compilation aspect of MPL - that's interesting.
(Of course, you could equally spin it that these are bad things - an exercise for the reader.)
The FSF takes a pretty major logical leap by considering dynamically linking a work to a GPL library to be creating a work that falls under the GPL.
Both EU and US scholars doubt that mere dynamic linking constitutes making a derivative work. (Specifically for the US, Galoob v. Nintendo ruled that a derivative work "must incorporate a portion of the copyrighted work in some form"; which obviously isn't the case with dynamic linking. - Legal scholars in the EU have come to similar conclusions when it comes to the various EU copyright directives.)
Generally speaking it's untested enough ground to kinda avoid the GPL for this usecase anyway, but the FSF's Legal FAQ presents things as fact in a way mostly only benefitting their cause.
It would take a very special situation when a company would rely on fulfilling all those conditions in order to use that as a legal defense.
A key question is if different aspect of a work should be considered separate independent works communicating with each other or as a single copyrighted work. In games people often talk about DLL files (in terms of modding), game content like images, video and sound, game engines, game and sever code. How much and what aspects can be modified without the permission of the copyright author?
There are generally three arguments I have heard in favor of a "single work". One is that everything will eventually be copied into memory, and thus while independently they may have individual copyrights the combined work which the author calls "The Game" is a single work.
The second argument is that all this technical details doesn't matter for a judge or jury. What matter is what those people perceive as a single work. Technical aspects like did the copying arrive there through the internet, a CD, a DLL file, or what have you isn't that important in determining the question about a single work vs interoperability between different independent works. It is all about the experience for the end-user.
The third argument I hear is that DDL files or programs that dependent on them are not independent. One can not run them independently, they are generally not developed independently, nor can the "single work" even start if parts are missing. Putting files into DDL is just a form of splitting the work into multiple files for technical convenient reasons, which is not a basis to form a legal distinction between a single work and multiple independent works. If the technical aspect would allow this then anything sent over the internet would loose copyright, since content is split into thousands IP packages which individually might not be large enough to be copyrighted.
As for a realistic example as to where this applies, I'd pick the age-old "GNU Readline" library. Readline is infamous for the fact that it's a standard library on Linux distros (because it's a bash dependency) that is easy to accidentally import in a C project and lands under the GPL. The FSF from what I can tell loves to parade this library around as a way to "gotcha" developers; it's to the point where even Readlines Wikipedia article mentions this[1].
In the case of EU law - this is just straight up not an issue[2]. As long as you're not distributing your software with readline, but rather with a dynamic link to readlines .so file (which for Linux can be easily assumed since it's a bash dependency and the overwhelming majority of Linux PCs have bash installed), readline's license doesn't apply since a user can just supply their own library and as long as it's compatible, it will work. It's hard to argue someone is distributing readline or making a derivative work from readline just by linking with its public API.
To put it in a slightly different form - the idea of linking not being a derivative work to stop somewhere because otherwise the literal Linux Kernel would force every program ever written for it to be under GPL2.0-only (which obviously isn't true, not even in US law from what I can tell), since every linux program is technically a derivative of linux the kernel. The EUs interpretation seems to be that it ends exactly on the moment the code in a program stops being ran from the files with which it's distributed.
---
Game mods are probably split down the middle, if we just look at them "as code" (so without going into asset patches - those would probably be a derivative work regardless, I'm thinking here of say, editing a loot table in a game; basically just number tweaks). Games with officially used mod loading can likely claim that mods are plugins, which would make them derivative works. That said, most games as of recent don't ship with mod loaders and rely on patching a DLL file shipped with the game[3], which likely would make an individual mod not a derivative work, given it's just an interface re-implementation with user-defined side effects.
Then you have the really old-school IPS files which just are straight up binary patches. I have absolutely no clue how those fit into the mix, given an IPS patch is literally a series of data offsets + what data to dump at those bytes. Those mostly fit with old ROMs though since IPS patches were abandoned due to inherent size limits + a magic word bug.
---
That said, ultimately it's important to keep in mind that law isn't computer code. It's not that if function foo takes argument bar and produces result foobar, that you always get result foobar with the law[4]. Not even in the US, which almost always defers to precedent ("case law") is that the case, and even less so in the EU where precedent is just treated as another argument rather than something to defer to. There's a zillion edge-cases to each example and a judge can rule differently in the end for most situations.
This is simply what the EU has written on the matter and from what I know about CJEU rulings, the CJEU tends to side on the interpretation that unless the goal is extremely blatant copyright violation, it's probably fine.
[0]: https://joinup.ec.europa.eu/collection/eupl/licence-compatib... (see: More details on the case of linking section)
[1]: https://en.wikipedia.org/wiki/GNU_Readline#Choice_of_the_GPL...
[2]: Full disclosure, I am not a lawyer, please ask an actual lawyer for legal advice.
[3]: Bepinex is the one used for the majority of Unity games and is one that jumps to mind immediately.
[4]: And that's generally speaking a good thing. Application of the law does require nuance.
A driver that itself was intertwined with the internals of the kernel, or would conflict with the exploitation of the linux kernel, or cause conflict with the legitimate interest of the copyright holders, might be a derivative work. The linux kernel has published clear boundaries for this (https://www.kernel.org/doc/html/latest/process/license-rules...), like the the syscall interface. Drivers that do not respect those boundaries may be more likely to fall outside of the EU law, but as with most legal discussions it would be a gliding scale.
Readline is one of those more odd theoretical case where the specific main work is the library itself. This makes questions like "legitimate interest of the copyright holder" and "normal exploitation of the program" a bit more complex. It would however never become a real legal case since anyone accused of infringement can just replace the small library with an compatible alternative and stop distributing the GNU Readline library. Since compliance is generally the goal by FSF I doubt they or any company would be willing to spend money on lawyers to fight over it.
In contrast, Unity sells their game engine library, so companies that tried to bypass copyright (by not paying) and "link" their games with existing installed versions on users computers would likely still end up in court. My money would also be on unity winning that battle.
But since GPL and LGPL are compatible licenses, can't the non-free author just fork the GPL to an LGPL version and then use it? A bit of inconvenience and technicality involved but still a workable workaround.
In an extremely simplified version, LGPL says you can only use the software if you guarantee A and B, while GPL says you can only use the software if you guarantee A, B, and C. Since {A,B} is a subset of {A,B,C}, licensing the LGPL software under something that requires A, B, and C guarantees A and B and so is fine by the LGPL. However, since the LGPL doesn't require you to guarantee C, then licensing software under the LGPL will not maintain all the requirements you must maintain to use GPL software.
Those that want to take the work of others for free, get the same payment that they are willing to pay upstream developers for.
Otherwise they can dynamically link it and take it as it is, or if it doesn't suit them, pay for the commercial license instead, and share their gold coins with upstream.
EDIT: missing words
Incorrect.
https://www.gnu.org/licenses/gpl-faq.html#LGPLStaticVsDynami...
If you statically link against an LGPLed library, you must also provide your application in an object (not necessarily source) format, so that a user has the opportunity to modify the library and relink the application."I believe I did, Bob."
I'm being more than a bit mean here, in my attempt to be slightly amusing, because there's actually an obvious workaround: dynamically link with the LGPL library, like people usually do, and then it's nice and easy. The sort of systems where this would be difficult are the sort of systems where actually distributing a GPL program is just going to be annoying anyway, and you're probably better off not even trying.
But it is actually an interesting idea! I assume for the average program you'd be including more of your symbol information than you might like, though, as object files have to find their external symbols somehow! (I imagine LTCG will add a lot of additional information as well. All this would add up to a lot of useful info that would assist in the sort of reverse-engineering effort that proprietary software vendors would like to make more tedious, rather than less so.)
But, if you really wanted to do it, and didn't mind putting a bit of effort in, you could probably do something. An enormous non-LTCG translation unit containing all of the code, probably.
This is false. You can static link as long as you follow https://www.gnu.org/licenses/gpl-faq.html#LGPLStaticVsDynami...
Additionally, LGPL3 can be tweaked to add an exception to allow unrestricted static linking. This is like... exactly what ZeroMQ did.
It's indeed an intuitive library, but it's far, far from perfect, thus why it has been rewritten countless times by Sustrik (crossroads), then by D'Amore (nanomsg).
The state machine in 0MQ is a nightmare of maintenance and a constant source of tricky bugs. The overall "all sockets in one" design makes things such as reconnection and peer identification pretty much impossible.
Don't get me wrong, there's a lot of good in 0MQ. But calling it a jewel that will be in use for decades is far from reality.
Was that sentence finished?
It's a task that you have to do, since the basic guarantees of 0MQ are almost never enough and is very well explained in the guide. But you have to read it carefully.
-CEO, iMatix Corporation sprl -23 April 2016
By October 2016 he was dead.
This PR basically merges his final commit.
I find it very sad.
I say all that to mean that anyone being sad about his death is something that Pieter would not have wanted.
Confessions of a Necromancer is his work biography, I was totally caught off-guard how relatable his stories turned out to be.
After reading this I wanted to read everything by him - which I am still in the process of.