Hacker News Reader

Comment by eric_b | Hacker News Reader

So they're moving to a new "community based", "light governance" model. [1]

There are plenty of problems with BDFL-style projects, but I think there are a lot of advantages too. Redis is unique in my experience in that it works the way you'd expect - it doesn't cause outages, it is fast, and it has a vanishingly small number of gotchas. The feature set is well curated and for the most part fits together cohesively.

The most important thing Antirez did, in my opinion, was to say "No" to things. No to new features that didn't make sense. No to PRs that didn't fit the vision. Saying no is one of the most important jobs of a project maintainer. It's a thankless task that upsets a lot of people. But it's critical for a project to stay successful and achieve the leader's vision.

Maybe I'm a pessimist, but I predict after a few years of this new model we'll see weird features, more stability issues, and performance regressions as more cooks enter the kitchen. Time will tell.

[1] https://redislabs.com/blog/new-governance-for-redis/

View on HN

> The most important thing Antirez did, in my opinion, was to say "No" to things. No to new features that didn't make sense.

Redis has had tremendous mission creep over the years. It started of course mostly as a volatile cache but I've now seen it also used as a message bus (in three different ways: BLPOP, PUB/SUB and Streams), for service discovery and as a general purpose database - something that Redis Labs (in my opinion: wrongly) encourages.

Memcache existed in 2009 when Redis was first released, also as a volatile cache...and still is just a volatile cache. Memcache is what "saying no" looks like. Redis is what "saying yes" looks like - and there are a lot of gotchas.

Well, kinda. Feature list does look like basically random assortment of vaguely related things (or "toolbox" if you want to be nice).

But... things that are also just useful often and for what you'd need to spin off a separate service for, or make suboptimally in whatever (No)SQL you use to store your data.

So it is perfect if you just want to prototype whether given approach is viable, and "good enough" for many apps.

Memcache is what saying no to changing the use-case looks like. Redis is what saying no to changing the architecture in order to implement features looks like.

Redis has stayed the same, architecturally, from the beginning: it’s a keyspace where the values are arbitrary in-memory objects owned by their keys, and where commands resolve to synchronous function-calls against the objects (or ref-cells) at those keys.

Anything that can be done without changing that architecture, is in-scope for Redis. (Though if a data structure doesn’t have wide use outside of a domain, it’s best left to a module.) Anything that cannot be done without changing the architecture, won’t be done.

Much of the “fun” I’ve personally had in watching Redis evolve, has been seeing how Antirez has managed to solve the puzzles of getting features that intuitively wouldn’t be a good fit for this architecture, to fit into it anyway: changing technologies that are implemented one way everywhere else, into something else for Redis, that still work, are still performant, and still solve isomorphic use-cases—if not with the same steps you’d use to solve them in other systems. (E.g. using Streams vs. using a regular MQ; using Redis Cluster vs. using regular RDBMS replication; etc.)

Would you mind elaborating on some of those 'fun' items - part of the problem of being aware of certain technologies without really 'following' them is understanding why they change.

Memcache also is about saying no to changing the architecture. That chosen architecture is similar to Redis Cluster -which is a change of architecture Redis did undertake.

I personally have not enjoyed all the strange and wonderful ways people have found to use Redis. Most of them are pretty fragile and (most dangerously) they often put all Redis uses in the same instance...with eviction turned on.

Wow you totally got it. This is exactly the idea I was using, I even saw an opportunity when the Redis model was potentially able to solve a new use case.

It didn't really start as a volatile cache, it started as an in memory data structures server. It's useful for all kinds of stuff memcached isn't (and has been since day 1).

It started as a memcached with persistence to disk for fast startup.

No, it was always about data structures.

Lists, hashes, sets and such that you have available in your programming language, but available as a networked server so any language/app/process can access the same data using the same useful structures.

This. To me, Redis was always about the rich set of operations you can atomically perform on data without going full ACID. It's very different from a pure cache (although it's probably a good choice if all you need is a cache).

Yeah. I do think that Redis has suffered from some mission creep, including a deprecated misadventure in to a mode where values could be stored on disk and only temporarily cached in memory.

Maybe that's what Cal and his colleagues were using it for at the time, but the claim that it started as a volatile cache is just not true. Interesting datastructures and the fork based persistence model were there from the start.

> but I've now seen it also used as a message bus

It works excellent as a message bus, though. And you can add HSET/HDEL to your list as the fourth way.

Strongly, strongly, disagree that Redis works well as a message bus.

Lots of reasons why but here is just one because I'm short on time: all of these (four) ways of doing a message bus with Redis offer either a work queue mode or publish/subscribe - but not both (perhaps excepting streams which I've forgotten some of the details of). I have yet to see many real world messaging scenarios where you don't end up using both.

A "proper" message bus needs to have topics, exchanges and queues - like Rabbit. 50% of that functionality is usually not 50% of the value but 10% or less (especially when you consider all the other things about Redis you need to think about - like how eviction is configured).

This is a bit typical of Redis: for any X, you can normally do X in Redis, but is it really a good idea to do X in Redis? IMO, usually no.

Name of the hash is a topic. Keys are nanosecond timestamps, values are the packets. You put in the data, you send a pub notify to the listener, listener goes and drains the queue. It acts as a direct exchange queue, recovers fine from a restart and it works wonderfully. For fanout, you can simply use pub sub. For topical, you can keep a list of direct exchange queue hashes. I haven't needed it.

Redis Streams is built-in and does all that for you. https://redis.io/topics/streams-intro

You're probably right. I did this before streams were a thing.

There's a wide field of messaging from distributed logs to queuing systems to event sourcing to complex service buses. Kafka, Pulsar, NATS, RabbitMQ, Tibco, NServiceBus, ActiveMQ, Solace, AMPS, and dozens more. What's "proper" depends on your needs.

Redis Streams actually does exactly what you describe and is basically a single-node fast Kafka replacement with multiple independent consumer groups, per-message acknowledgements, and queue semantics. We use it successfully in many high-load scenarios.

> There are plenty of problems with BDFL-style projects, but I think there are a lot of advantages too.

BDFL governance is extremely underrated in the opensource world. People attribute the success of the Linux kernel to its open contribution style, but I will argue that Linux is successful because there's a dictator at the top that enforces a direction, a long term goal, and most importantly, says NO.

Community based governance is a direct cause of that jwz's Cascade of Attention-Deficit problem one can find all over the open source world, especially Linux desktop related.

Does FreeBSD have a BDFL?

We don't have enough data points to determine if what you are saying is correct. I have also seen lots of BDFL projects smoke out.

Is the lesson that actual dictatorships only survive long term if 1) the dictator is a nice person and 2) they eventually get managed by an open democratic process?

I think you are actually describing the two clocks problem. Lots of BDFL projects have their own Cascade of Attention-Deficit issues, the project only goes in the direction the BDFL actually cares about.

A (software) dictator is not necessarily good. But let me double down and say that a community cannot create a product as good as a single head at the top can do, for the single reason that people care only about their own garden, and integration needs someone to oversee and be responsible for all parts.

There is no Steve Jobs in a democratic committee.

Cannot seems rather strong. I haven't been paying very much attention, but I was under the impression that Postgres doesn't have a dictator?

Or Rust, which - whatever you might think of it - does not feel very designed-by-committee.

Rust governance is more amazing than Rust the language and that is saying something.

It's evolved. There was a point that Tom Lane was the last line of defense against anything that strayed too far.

Heck at one of Postgres conferences there was a shirt that said something like "Tom Lane reviewed my patch and all I got was this stupid shirt."

While he was not a BDFL style person, in the past, if he didn't like a change, it didn't make it in.

Here’s the thing: you need both. You need breadth AND focus. Breadth tends to come from the community and focus from a small number of individuals. But both are important.

Keeping both of those balls in the air at once is why being a software coordinator like this is such an exhausting job.

As usual, Fred Brook's Mythical Man-Month had some valuable insights on this back in 1975:

> To make a user-friendly system, the system must have conceptual integrity, which can only be achieved by separating architecture from implementation. A single chief architect (or a small number of architects), acting on the user's behalf, decides what goes in the system and what stays out. The architect or team of architects should develop an idea of what the system should do and make sure that this vision is understood by the rest of the team. A novel idea by someone may not be included if it does not fit seamlessly with the overall system design. In fact, to ensure a user-friendly system, a system may deliberately provide fewer features than it is capable of. The point being, if a system is too complicated to use, many features will go unused because no one has time to learn them.

* (This is quoted from the Wikipedia page as I don't have an online copy of MMM to hand).

this is IMHO no different the real world - a 'good' despot/monarch/etc can be a good leader, but this is highly dependent on the situation (read: leader), and leadership transition process is always a risky one. Democracies/Republics work well so long as the community has a generally shared vision, but can be hampered when there is too much dissent, but generally avoid the succession problem

OT: Indeed, a good dictatorship is better than any democracy, but it doesn't work in practice because for every Augustus there's 10 Caligula.

Democracy is just the least worse political system.

Democracy is more of a principle (is just means that legitimacy of government comes from some aggregate choice of the governed), than a political system. Radically different political systems can be classified as democracies, all the way from direct vote (self-organized communes, also Switzerland) to complex federations with layers of indirect representation, such as the USA or the EU -- with a lot of other interesting alternatives such as sortition.

In a FOSS it is very easy to fork, so if the BDFL get's mad you can just say thanks and fork the project and continue. So that keeps BDFL in check.

IRL you own a plot of land, or a home. Or you have a small factory, and there are a lot of network dependencies, who does you sell the supplies, where you sell your products. It the local BDFL get's mad it is very difficult to move. (Specially if there are some cash flow controls were you can't get out with the money that you get selling your home or that you have saved.)

You can fork Linux if the dictator doesn’t suit you, but you can’t fork a nation. You also can’t switch to another nation that is more appealing, generally. Finally, Linux can’t imprison and torture you if you criticize it

If you're comparing FreeBSD to Linux then I think that says a lot about the merits of BDFLs, really.

Did FreeBSD not have the discussion about this years ago? Though they did not have a single person, but a 20 person core team, which were cut down to a 9 person core team that is voted for every 2 years.

http://freebsd.dk/sagas/freebsd/#id1

Sounds similar to debian, which is also a very successful and stable project.

The core team doesn’t really make technical decisions. It’s more for administrative/social/governance type stuff.

Who would then make the technical decisions? The owners of the different sub systems?

There's no hard rule. The committers are expected to behave like a civilised human beings, ie talk to each other to reach consensus.

Let's say you have a fix for some driver used on ARM. You'll submit it via Phabricator to get ARM maintainers to review it. If they don't, because eg they don't have time, then you'll probably commit it anyway, but if someone points out you did something wrong, you'll be expected to fix it.

The main differentiation (and risk) with BDFL projects is having (or not) a very competent (like one percenter) BDFL.

The structure of a committee project doesn't make it better or worse than BDFL. It just makes it more average. Any greatness present is toned down. And substandardness is brought up.

This also main reason the "led by greatness" BDFL projects are better than the committee run ones and the "led by less than great" BDFL projects "smoke out". The committee's tend towards the middle of the bell curve, BDFL's are all over the place.

>Does FreeBSD have a BDFL?

No, but it doesn't have as much success as Linux either

I think it's _just_ underrated; I think it's underrated in the business and government worlds too. A good leader should take different points of view into account, but should also be able to make a decision when needed.

> I will argue that Linux is successful because there's a dictator at the top that enforces a direction, a long term goal, and most importantly, says NO.

Linux is made because of many people all over the world, but it is successful because of the BDFL at the top. A consistent sense of mission absolutely benefits a project in the long term. There just isn't anyone that's going to get your idea better than you, when you're doing something that you know enough about to seriously get started on.

Underrated in the open source world — whilst at the same time, feels obvious in the writing world?

Imagine an author has gotten started writing volumes 1 and 2, and those volumes became popular — maybe the Harry Potter books as an example. Now, adding more authors with equal "story-line decision making power", or a committee, for volumes 3, 4, 5, is a weird idea, right.

Look at what happened with Game Of Thrones! Total disaster at the end that ruined the whole thing for everybody.

I feel like Antirez has already done everything needed to ensure this doesn’t happen, by 1. building the module system into Redis, and then 2. working with companies like Redis Labs to establish that Redis is enhanced through these modules, and to ensure that modules offer everything required to enhance Redis in all the ways that matter.

In essence, Antirez has protected the core of Redis from needing to bloat, by making it possible for anyone who wants to build on top of Redis to literally do so—build on top, rather than inside. As such, I expect future PRs to Redis Core to look much like current PRs to Redis Core: just fixing bugs, adding stability, and resolving infrastructure-level interoperability concerns.

I 100% agree with you and I think it explains why most OSS UI projects are, well, terrible. Questionable features, inconsistent UX, millions of "options"... these are the hallmarks of a community model lacking strong leadership and a coherent vision.

As for the future of Redis I choose to be more optimistic. I think the Redis philosophy (thanks to Antirez) has become ingrained. The benefits of a minimal feature set are hopefully well-established.

In the very least I suspect this will last much longer than many expect before the bureaucrats take over.

BDFL: Benevolent dictator for life https://en.wikipedia.org/wiki/Benevolent_dictator_for_life

(This is also referenced and linked in the article.)

> The most important thing Antirez did, in my opinion, was to say "No" to things.

This is hard. It makes you feel bad inside to tell people "No", especially when they open PR and clearly did a lot of work adding some feature.

Thats something a have regarding PR and code reviews: much frustration can be avoided by discussing what to do and how to do it before showing the work done (avoiding also infinite bikeshedding). Often code review talk about the strategy to do X and not the way to do X

And some people take the 'No' badly. It's frustrating having to explain that the code in a PR needs to be maintained forever, and it's the repo maintainer that will have to do it, not the contributor. A simple proposal upfront makes all the difference, and saves human effort and feelings.

>Saying no is one of the most important jobs of a project maintainer. It's a thankless task that upsets a lot of people. But it's critical for a project to stay successful and achieve the leader's vision.

Reminds me of Brian Goetz: "We have to say no to almost anything. If we didn't, things would pretty quickly degenerate to the point where the system collapses on its own weight (...) if we did this, which of these eighteen possible features might we foreclose on, and might we want to leave those possibilities open, at the cost of saying no to this thing here." (https://www.infoq.com/presentations/java-history-present-fut..., 24m+)

The one drawback of the BDFL model is that you need a person, and a very specific kind of person at that, which tends to be irreplaceable - to sustain it.

Redis has succeeded because of antirez. I really wish it can continue doing so for the long run without him. More than anything it reminds me of Steve Jobs' death (although gladly this is in much happier circumstances and I wish antirez many more decades of hacking). It's true that a decade later Apple is still doing great, but you just know things would have been different if he was still around.

Instead of a single BDFL, you could also have a board of e.g. 3 BDFLs. Those 3 people would have the final word, but they rank equal, and have to come to a common conclusion, e.g. by discussing internally. That only works if the board is not too big, and the people are somewhat compatible to each other. But in that case, I think it works much better, because there is less pressure on a single person, also less work (they can distribute it somewhat), and probably more reasonable and consistent conclusions.

It seems that this model is not so common? At least the list on Wikipedia (https://en.wikipedia.org/wiki/Benevolent_dictator_for_life) only lists Django as an example with 2 BDFLs.

I was active in a smaller open source project (http://www.openlierox.net) where it mostly was like that, i.e. we were 3 main developers, and all major design questions were discussed among us. But this example is probably not so representative, as there rarely were further contributions by other people. But anyway, I was very happy with this model, that there is not a single leader, but 3 people.

I think the mixed historical record of consul republics might be relevant to why a single leader is often a better idea than 2-3.

I'm thinking that a project that starts out with 3 BDFLs (like yours, @albertzeyer) and works fine & builds sth people "love" — it'll continue working fine with those 3 BDFLs.

And a project that starts with just 1 BDFL, and works fine & builds sth people "love", it'll continue working fine with that 1 BDFL.

If, however, replacing one of those 3, or adding another BDLF to a one person project — I'd think that by default, that won't work.

... Unless the new person has worked next to the earlier BDFLs for years, so the previous BDFLs can be ok certain s/he is "the right one"?

Whatever works fine, works fine — the risky thing, is introducing a change (a new person)?

Thing is, with a consul or triumvirate system (you can’t have more than one “dictator”; all of these terms are from Ancient Rome anyway so let’s use them properly), if any disagreement arises between the 2 or 3 leaders, it will develop a new instability where there wasn’t one before. Or else, one of the 2-3 leaders will start to dominate the others. Even without turnover the dictator system is more stable. In fact the only benefit of a consul or triumvirate system might be the stability and continuity under turnover.

Do you have any tips for explaining to stakeholders that things in a Redis cache expire on async schedules?

Reply on news.ycombinator.com