ApsaraCache: A Redis branch originated from Alibaba Group | Hacker News Reader

ApsaraCache: A Redis branch originated from Alibaba Group

github.com

43 pointslaithshadeed41 comments

Hi all, I'm here in Hangzhou and had the pleasure to have a meeting with the team working at that. It's a great team and we'll work into merging stuff into upstream Redis when it makes sense. It's cool that companies other than Redis Labs are starting to contribute to the development!

Note that this branch is based on Redis 2.8 so of course porting stuff requires some work.

Can I ask what you think of pedis ? (see my other comment on the thread)

Hey, I never tested it, I've no idea. In general I think that threading Redis is a good idea and I'm going to add more threading very soon. I'm less sold on making disk the Redis primary storage...

>Note that this branch is based on Redis 2.8

Hmm. The Github repo says it's based on 4.0: "ApsaraCache is based on the Redis official release 4.0" (from https://github.com/alibaba/ApsaraCache/blob/develop/README.m...)

Oh probably I mistaken other patches I discussed that will need to be ported. Thanks for the reply.

Reminds me of HHVM: a specific solution developed by a specific team for their specific problem.

Might help Redis on the long run though: some ideas might be brought back to Redis, or at least it will bring some challenge to the devs.

Meanwhile, keep calm and use Redis ;-)

Agreed, though HHVM went on to be deployed at some other small websites [1]. Maybe this will too.

1. http://hhvm.com/blog/7205/wikipedia-on-hhvm

On the other hand, Wikipedia is going to have to migrate back off of HHVM now that Facebook is planning to break compatibility with PHP and focus totally on Hack [0]. So hopping on board someone else's solution can bite you if their priorities change.

[0] http://hhvm.com/blog/2017/09/18/the-future-of-hhvm.html

> Facebook is planning to break compatibility with PHP and focus totally on Hack

Ok...

> Wikipedia is going to have to migrate back off of HHVM

Why?

For years, HHVM had a significant edge in performance and a significant handicap in compatibility. Today, PHP7.1+ is ~at parity in performance but the community still has not fully embraced HHVM. In fact, some flagship projects like Symfony even stopped on purpose. Increasingly, the main reason remaining for running PHP on HHVM is if you really want static typing - and this is not a natural preference for most PHP teams.

Hack/HHVM is breaking compatibility with PHP. MediaWiki is a large PHP software project. Other MediaWiki users run the software in non-HHVM environments.

30% faster by which measurements? Not much info on the docs....

yeah, one of the few red flags

> In short connection scenario, ApsaraCache makes 30% performance increase compared with the vanilla version.

sounds like a cherry picked case to me

Cherry-picked IMHO isn't necessarily the right label for that. They don't claim they made an universally 30% faster redis, they clearly say one of the things they did improved performance for a specific case.

If anything, the OP here is misleading by quoting it as a general improvement.

for them it is a real need.

(Talk with Daniel Zang CEO Alibaba Group) https://youtu.be/YhlvtcBwxPM?t=326

I just tested it with redis-benchmark. The performance is the same as with Redis.

Oh...

It mentions "in short connection scenario".

I take that to mean repeated connect/set-get/disconnect. Like you might get from a client in a fastcgi scenario...php for example. Maybe add the -k 0 flag to the benchmark and a low-ish number of concurrent clients.

This looks pretty interesting to me. But it leads to a lot of questions that are unanswered on the webpage.

The claims of 30% faster for a particular workload are reasonable, but what configuration is needed and what exact workload are we talking about? I'd love to see some benchmarking code.

What are the limitations imposed by running in memcached mode? Can you still do clustering?

What other changes are they planning? Site just says "many". Are we talking about memory allocation or persistence changes? Maybe additional interface compatibility modes? It's hard to bet on this when we don't have any idea what else it will bring to the table.

Where possible, I'd really like to see these improvements factored into distinct redis modules and made available as plugins to the core.

Goals: For both support Redis and Memcached protocol with no client code need to be modified. For highly stable and efficient in production environment.

At face value it appears to be a NIH driven development. It is not only a redis fork it is also a persistent memcached port, which is _only_ selectable on launch. Maybe there is a catch, but lack of a roadmap and/or design document certainly doesn't really help.

I remember some time ago Alibaba was the largest Memcached user on the planet, may be they wanted to smooth the operation and use Redis whenever possible?

Why forking instead of submitting pull requests to Redis?

It seems they made a lot of changes, and some would be probably unsuitable for Redis. "Memcached mode" for example.

The readme states they have a few extra unreleased features.

I'm guessing their changes are for internal use and may be out of scope for redis or simply they want to share their internal fork and get feedback/help.

I know you didn't say anything to the contrary but I feel it needs to be said: forks aren't inherently bad, they're a sign of a mature and healthy open source ecosystem.

And I'm sure if it's worth it to someone, perf improvements will eventually get upstreamed.

I wonder if Redis throughput is even a problem in practice. People usually add nodes to get more capacity (memory).

For most use-cases, Redis nodes will saturate network IO before CPU, but it should be noted that Redis is single-threaded so YMMV.

We do heavy pipelining and tend to max out the single thread, so we run multiple redis instances per host.

One of the most useless FAQ I saw. I would have been more interested in knowing the improvements or changes made when compared to Redis. I had some trouble with large scale master slave Redis configuration. I was hoping if they have solved the issues.

They list the two things they changed compared to redis in the readme.

I went back and read again. "ApsaraCache is based on the Redis official release 4.0 and has many features and performance enhancements" - I could find only memcache and performance improvement. Now Membache has nothing to do with Redis, its something not improving Redis, and 30% performance improvement is very vague statement with no other supporting benchmarks and documents. When they say many, I can't see any other than these two.

They implemented the memcached protocol on top of redis.

Also,

> There are many features in ApsaraCache, the following two are included in this release and the other features will be gradually released in the subsequent, so stay tuned.

I think they are also working on: https://github.com/fastio/pedis (redis implemented on top of seastar-framework / scylladb )

any experience using it in production? looks promising results

still missing many features so I don't think it's ready, you should ask on their issue queue

Diffs of what they changed in /src from Redis 4.0: https://github.com/asaradiffs/apsaradiffs

documentation and readme of alibaba projects are scary. Have anyone use this or any alibaba open source projects for production?

Does it still run on only one thread?

This project is some kind of a joke;

Look at the commit history; https://github.com/alibaba/ApsaraCache/commits/develop

15 are "writing of README / adding my copyright", and 2 are actual feature (and not long)

I think this is what you are missing: https://github.com/alibaba/ApsaraCache/commit/07518063b17b13...

Kind of sad how things can't be partial and in the process of being open-sourced, no they have to be complete or they are "jokes".

Please don't post uncharitable dismissals of other people's work to HN. If you read https://news.ycombinator.com/newsguidelines.html properly you'll see that doing this breaks more than one of the site guidelines.

Reply on news.ycombinator.com