Redis re-implemented in Rust
github.com
github.com
I also have no intention of making this project live as long or have as many users as Redis does.
Can rust be readable?
https://github.com/rust-lang/rust/blob/master/src/libstd/col...
You can also look at `mio`, the asynchronous IO library that's popular in Rust:
https://github.com/carllerche/mio/blob/master/src/notify.rs#...
fn search_entry_hashed<'a, K: Eq, V>(table: &'a mut RawTable<K,V>, hash: SafeHash, k: K)
-> Entry<'a, K, V>In plain English:
`search_entry_hashed` is a parametric function. It works in a type safe way on any kind of K and V. It takes as arguments a reference to a mutable hash table, a hash function and a key, and returns an `Entry` object whose precise type depends on the type of the parameters.
You could hardly express that more succinctly. You can't understand it by skimming it, but it is a complex definition.
fn search_entry_hashed<'a, K: Eq, V, H: SafeHash>(table: &'a mut RawTable<K,V,H>, hash: H, k: K)
-> Entry<'a, K, V> fn search_entry_hashed<'lifetime, KeyType: Eq, ValueType>
(table: &'lifetime mut RawTable<KeyType, ValueType>, hash: SafeHash, key: KeyType)
-> Entry<'lifetime, KeyType, ValueType>- maxmemory key eviction
- hash values
- ~2/3 of the set operators
- multi/exec
- lua scripting
This is an interesting and potentially useful effort, but a replacement for Redis it is not.
I'm sure pull requests to bring it up to feature parity would be welcome!
Its a very interesting piece of work though.
I'll be interested to see Antirez's view on the trade-offs between C and Rust for this.
http://redis.io/topics/data-types-intro
The data structures are all addressed by string keys.
Redis can persist this heap to disk, and load it again, so you get a measure of durability, but the typical use case is for data you can afford to lose - caches, metrics, some kinds of events, etc.
Redis's key non-functional strengths are speed and robustness. Operations people love it because you stick it in production and it just quietly keeps on working without needing attention or setting your CPU on fire.
To my mind, any project should have PostgreSQL as its first data store. But it should probably have Redis as its second, when it finally has some data that needs to be changed or accessed so fast that PostgreSQL can't keep up.
(Kafka is third)
It simply means that the key-value store is directly loaded into the memory (RAM) and is available for fast access, but the data is retained (persistent) even after the application is closed.
It is usually used as cache store, queuing messages to communicate with different processes locally or distributed.
Its data structure cover a good part of what you'd need with generic data structures, which makes Redis an easy way to do the logic of, say, List intersection of friends common between multiple people, sorted set of goods ranked by their amount, all of this shared with other processes.
Redis also offers pubsub capabilities in two forms:
- A standard PUB/SUB couple which does what you think it does
- Blocking pop on a list for a client, and a push for another client, which will "wake up" the first one with the value.
It's a very versatile swiss knife.
It can be used for caching, queues and for applications with volatile data.
https://github.com/frankmcsherry/timely-dataflow
https://github.com/frankmcsherry/differential-dataflow
Or, just tell your friends. :)
Better yet, write some python / pandas / dataframes / whatever_the_cool_kids_need layer on top, and rule the next big data drama cycle.
You are right, though, that if the processing is extremely "batchy" and all data dies at the same time, then it doesn't make a difference.
I'm not convinced that's the reason why Java is used for it. There are native alternatives like HPCC which claim to perform better.
As was noted, concurrent access to shared data is not something very common in such distributed computation scenario. Well designed processing will avoid it, and thus will avoid need for locking as well.
The goal with performance is almost never to get the maximum possible performance but the best peformance/effort ratio for your needs. This is true even for performance-critical applications. As there are diminishing returns, every increase in performance costs more than the previous. Very, very few applications are willing to invest an extra 30-50% effort to go from 98% (of maximum performance) to 100%.
As to concurrent access -- see my other comment (depends on use-case).
I generally agree with you on GC but, assuming you haven't, I think you would benefit from actually trying to write some of these algorithms in a manually memory managed system. It's really not as hard as you consistently make it out to be in a language like Rust with very precise control over lifetime (which can be exploited just as well for the hazard pointer itself as it can for anything else).
1. The use-case I have in mind is an in-memory database because a. that is a significant part of what I do and am most familiar with and b. because in the age of large RAM, its most effective use is embedding large parts of the database within the application process (whether as an actual product, or replicating some part of a database's functionality ad-hoc). While doing so in an environment without a GC is certainly possible, it is both much harder and/or (depending on the effort spent) less scalable with increasing numbers of cores. I have come to that conclusion after spending many years trying doing that in C++.
2. I have no doubt that Rust -- and I take great joy in this fact -- makes a whole lot of things much easier than C++. But there is also no question that Rust is harder to program than most GCed languages. The question from my perspective, then, is not "why not do it in Rust?" but "why not in Java?". There is a good answer to that question: Rust is much more suitable than Java (in terms of performance and RAM footprint) when running in constrained environments. But unless you're targeting a constrained environment, why work even a little harder at all?
More generally, I believe that the most efficient form of memory management in unconstrained environments, in terms of both throughput and worst-case latency -- and I've had this discussion on HN with pcwalton -- is an environment that allows three types of memory: stack, scoped arenas (including a permanent scope), and a GCed heap. Real-time Java (RTSJ) provides exactly that. Combining arenas with Rust's beautiful scoped threads is a great idea. I believe that reference counting has no place in such unconstrained environments.
The folks I know doing stream processing over the JVM have garbage collection (and avoiding it) as perf issue #1. For example, you can read about Flink's approach to avoiding GC at
https://flink.apache.org/news/2015/05/11/Juggling-with-Bits-...
You can read a more scientific discussion of GC vs region allocation here:
https://www.usenix.org/system/files/conference/hotos15/hotos...
1. Memory with very clear lifespan is obviously easier to manage without a GC.
2. The fact that GCs are a performance issue, does not mean that using a non-GC environment would yield better performance. It is often easier to avoid GC in GCed environments than to handle concurrent access to data in non-GC environments.
I disagree that avoiding GC is easier; you're swimming upstream the entire time. Managing memory manually in GC environments is annoying because the platform wasn't designed for that.
As Frank said, you want to avoid sharing mutable data irrespective of GC or not - that won't scale well with larger core counts. Instead, you want to delineate ownership so you have single writer and many readers, and share the data that way.
> For systems where managing data is one of their reasons for existence, it creates a negative effect.
I think that people who author such systems/libraries know when to rely on the GC and when not to.
> GC'd runtimes also bloat memory consumption to support GC
When you write, say, an in-memory OLTP database (that's the use-case I know best), you keep the index -- that's where concurrency is managed -- on the heap and the user data off-heap. User data is the bulk of the RAM, but you only pay the GC footprint-overhead for the index.
> you're swimming upstream the entire time.
Going off-heap for large, known-lifespan data sets in Java has been widespread -- idiomatic, almost -- for at least five years now, if not ten. There are so many libraries that do that, that it's very, very simple. Again -- that's just where most data lies, not where the most effort is required. You end up with a very small percentage of your code that is not quite "beginner's Java" -- that's not swimming against the current; that's hitting a few strokes for one tiny section of the journey. It's managing the transactions and update to the indexes that requires 95% of the code, and where you end up thanking your lucky stars for a good GC.
I haven't measured exactly, but the entire off-heap code in our database is under 3%, even though over 85-95% of the data is stored off-heap.
> Instead, you want to delineate ownership so you have single writer and many readers, and share the data that way.
Even single writer/many readers still require a concurrent access mechanism. A GC tremendously helps with this pattern.
Besides, the best way to manage data -- and that's based on how caching mechanisms work at every level -- is single-writer/multiple-readers approximately, namely the single-writer may change identity at any time, but you ensure that it happens rarely. Once you try to make ownership permanent, system-wide locking is required. It's that flexibility that helps with amortizing the costs and making sure you're never hit with the worst case at every transaction[1]. There's some literature touching on that required flexibility, but here's my own take: http://highscalability.com/blog/2012/8/20/the-performance-of... (that flexibility is required to keep the B-tree evenly distributed across nodes -- they can be across the network or even NUMA sockets).
[1]: Changing the owner is the amortization pressure-release valve (all amortized algorithms work on that idea of slowly increasing pressure, that is then released seldom but regularly). It replaces sharding's rebalancing, which usually requires a stop-the-world lock (or most of the world).
There's also substantial amount of code around dealing with i/o efficiently (and not churning garbage), both net and disk (if there's any persistence involved in the product).
>I think that people who author such systems/libraries know when to rely on the GC and when not to.
Perhaps there's more awareness now, but there are plenty of JVM based systems where moving off-heap (and/or changing code to ease GC workload) came much later in the dev cycle.
>Going off-heap for large, known-lifespan data sets in Java has been widespread -- idiomatic, almost -- for at least five years now, if not ten. There are so many libraries that do that, that it's very, very simple. Again -- that's just where most data lies, not where the most effort is required. You end up with a very small percentage of your code that is not quite "beginner's Java" -- that's not swimming against the current; that's hitting a few strokes for one tiny section of the journey. It's managing the transactions and update to the indexes that requires 95% of the code, and where you end up thanking your lucky stars for a good GC.
IME, you end up not using a good bulk of JDK and 3rd party libs because they allocate frivolously. You end up having to look at internals of other code to see if they allocate or not since nobody in the java space cares about documenting that aspect (because allocation is mundane). If you then need to tune GC knobs, may the force be with you. On top of that, once you go off-heap, you're doing roughly the same style of block memory management that you'd do natively, except less efficiently (e.g. some Unsafe operations aren't JIT optimized as well, some operations aren't even intrinsics so you get JNI call overhead, you always get zero'd memory if you call Unsafe.allocate() even if you then overwrite everything, etc). You're probably going to now tell me that getting 90% of the performance is fine ...
I suggest you look at some non-managed OSS systems that deal with high-perf workloads. Your posts always make it sound like it's nigh near impossible to build anything of the sort without GC, which is ridiculous. If you want to see what a modern C++ rendition would look like that attempts to avoid sharing mutable data, have a look at https://github.com/cloudius-systems/seastar
That's more of your concern because you care about worst-case (very) low-latency domains. Short-lived objects are not a big concern for those who just wish to avoid the large pauses (and can live with the very occasional query taking 40ms instead of the average 50us).
> with i/o efficiently (and not churning garbage), both net and disk
At least one of the most popular Java IO libraries don't create much garbage (Netty 4), and besides, that's not a major concern at all because short-lived etc...
> you're doing roughly the same style of block memory management that you'd do natively, except less efficiently
Only for very simple lifespans. The heap data guards the offheap data wrt concurrency. And so what? All of it still amounts to a tiny, almost negligible, portion of the code, and that portion turns out to be quite simple. I'd rather write a small simple package for stuff that would be taken care of automatically by a non-GC language than write lots of complex, bug-prone code in a non-GC language for stuff that's taken care of automatically in GC runtimes.
> you always get zero'd memory if you call Unsafe.allocate()
But you don't allocate offheap slabs often at all.
> Your posts always make it sound like it's nigh near impossible to build anything of the sort without GC
It's much more work, and requires placing and reasoning about various constraints that may prove problematic. An example later.
> If you want to see what a modern C++ rendition would look like that attempts to avoid sharing mutable data, have a look at https://github.com/cloudius-systems/seastar
Thank you, I will. The example I like to give is that of VoltDB, a "NewSQL" database written in Java and C++ (C++ for the core) which uses internal sharding. Every thread is responsible for a slice of the data. But that database (designed or supervised by none other than Michael Stonebraker) requires locking for any join (even they couldn't have come up with a better way to do it). But their approach displays results that are far from groundbreaking (for modern, in-memory databases), and places a lot of burden on the user to carefully choose a sharding strategy. As another example, MemSQL don't shard, and use a lock-free index (they use concurrent skip-lists). They wrote the code in C++, so obviously that's possible, but put a lot of effort into just implementing the skip-lists (they use hazard pointers), and ended up with results that could have been just as good -- or better -- with a general-purpose GC. BTW, another thing they missed by picking C++, is that in order to make queries faster, they compile SQL to C++ at runtime, compile that C++ and load the resulting code. Java would have given them a much more efficient JIT.
> You're probably going to now tell me that getting 90% of the performance is fine ...
As a general case -- absolutely. In this case, Java gives you more like 96-97% performance for significantly less effort.
EDIT:
I've now read about seastar (and realized I know the people behind it). That approach is the absolute best way to get the worst-case lowest latency possible, if you're willing to pay a significant cost in design (how do you transactionally update data on two shards? -- you can't. Imagine a game object moving from one shard to another -- it will either show up twice or disappear momentarily). But if you're willing to relax the worst-case latency, you can gain significant gains in both throughput and ease of development (and still keep latency low enough for almost all domains). That's certainly how I'd design video-streaming or a network/phone switch -- not so much an online game, IoT sensor-fusion, or a group-chat app.
How big are the jobs being done? I remember reading a statistic[1] that is a couple of years old that 50% of the big data jobs were less than 10 GB. 90% was less than 100 GB and only something like 1% was bigger than a TB. You can fit multiple terabytes on a single machine. I wonder if anyone has new statistics about this.
[1]: http://www.msr-waypoint.com/pubs/204499/a20-appuswamy.pdf
I don't like the 90% argument. 90% of apps are just dumb frontends to a MySQL database, so you don't need Rust, or Spark. As you can see, that doesn't tell you anything meaningful about the other 10%.
I have good context on this from seeing the analytics warehouse for a very popular social network. Even with a multi-terabyte hot database, the analytics dataset fit on a single 2U server full of drives. I have not seen a deployment of Hadoop yet that makes sense. Usually I see things like a 2 or 3 TB log spread across 60 nodes and a job to count entries matching a regular expression (because Hadoop becomes the hammer hunting for nails), which can easily be done in a VM on a Mac in minutes.
That was the thrust of the original post, and I agree wholeheartedly.
Yes, I agree that it's not an uncommon situation. But regardless whether legitimately big deployments are common or not, I never really appreciated some negligence with performance in using Java for such things. Just because it scales, it doesn't mean that overhead should be ignored, because the cost of the overhead scales too.
So going back to my original point, having such kind of systems made with Rust would be really much better.
Could someone (or OP) elaborate on the value that re-implementing a whole software to a new language provide comparatively to just building an interface "bridging" both worlds?
To clarify, my metric for "value" is usefulness to other people. That is, without considering the (interesting) learning opportunity that it represent for the author.
For example, someone developed a Python interface to the Stanford Core-NLP library (written in Java). Would re-writing the Core NLP library to Python be useful to the community? How to figure what are people needs?
I am asking because while I think it would be ton of fun and allow me to learn a lot, I also value building useful software and re-writing a whole system sounds like an overkill except for a few very niche cases..
And if I am not mistaken you would need a team at least as large as the parent project to implement new features, fix bugs and keep pace with it. Looking forward to hear what HNers think!
edit: clarified ambiguities
To learn Rust.
First, I wanted to make it as pure Rust as possible. I tried to avoid UNIX specific code, and since there is no library with Windows support for asynchronous IO in Rust, I was pushed into spawning thread and blocking waiting for client data. I quickly noticed that the benchmark was way below Redis (around 60% of ops/sec with 50 clients). But then someone point out to me[1] that I was running tests in a machine with two cores, and this actually may be better for machines with multiple cores[2]. I have yet to try it out and benchmark the results.
So far Rust API was disappointing for network operations. For example, `TcpStream.read()`[3] and `TcpListener.incoming()`[4] do not have a timeout. Maybe because its development is driven for Servo and not for servers.
I have thought about doubling down on multithreading and instead of a global database lock as rsedis is using now, having one per key (or some other arbitrary partition), and having concurrent operations, which is hard to do safely in C. But I have not gotten there yet.
[1] https://github.com/jonhoo/rucache/issues/2
[2] https://github.com/jonhoo/volley/
[3] http://doc.rust-lang.org/1.0.0-beta/std/net/struct.TcpStream...
[4] http://doc.rust-lang.org/1.0.0-beta/std/net/struct.TcpListen...
> Maybe because its development is driven for Servo
> and not for servers.
The reason timeouts were dropped is in the IO/OS reform RFC: https://github.com/rust-lang/rfcs/blob/8fa971a670f9b9bc30f31...
> set_timeout has been removed for now (as well as other
> timeout-related functions). It is likely that this may
> come back soon as a binding to setsockopt to the
> SO_RCVTIMEO and SO_SNDTIMEO options. This RFC does not
> currently proposed adding them just yet, however.
> All timeout support is removed. This may come back in
> the form of setsockopt (as with TCP streams) or with
> a more general implementation of select.
That said, the lack here certainly hurts, and we did manage to stabilize Duration, paving the way for timeouts to return.
EDIT: Oh! I forgot that https://github.com/rust-lang/rfcs/pull/1047 got merged recently. https://github.com/rust-lang/rust/issues/25818 implemented it. http://doc.rust-lang.org/nightly/std/net/struct.TcpStream.ht... shows it implemented on nightly, so you can actually even do timeouts today, just not on the stable channel.
1. We keep open issues on that repo to track ideas.
2. At some point, someone may decide to formally propose an idea. They may or may not post a "pre-RFC" to internals.rust-lang.org to get early feedback.
3. An actual RFC will get filed at that repo as a PR.
4. The relevant sub team will check it out and comment, and at some point, consensus will be reached either way.
5. The RFC will go to 'last call' for a week, making sure all voices have been heard.
6. Assuming nothing in last call blocks moving forward, the RFC will be accepted.
7. The RFC will be implemented.
So, in short, yes, subscribing to that repo will notify you of the relevant discussions.
IRC is used for quick casual conversation. The internals forum is used for discussion, "pre-RFC", and the like. The rfcs repo is use for actually discussing formal RFCs, as well as wishlist language and library bugs. The rust repo is for the actual implementation of rustc and the standard library itself.
Seems like those parts still need more work.
There are also libraries like mio that can give you no blocking IO. Rust's standard library is deliberately minimal, exploring the ecosystem a bit is helpful, though admittedly not as easy as it could be. Working on it!
Making it safer and even catching bugs in the original implementation (both things Rust will help with)?
Making it integrate seamlesly with the new language's ecosystem? E.g. Lucene is Java, and someone could use that, but there are tons of ports of it in C, C++, Python etc, providing convenience to integrate it with projects in these languages.
>And if I am not mistaken you would need a team at least as large as the parent project to implement new features, fix bugs and keep pace with it.
Not necessarily. A project with 10 part time contributors could be matched with a project with 2-3 full time competent hackers for example, or even surpassed.
There used to be several ports, though most are dead and/or are several major versions behind. A new C++ or Rust port would be great, though unrealistic given the huge project side.
It could be done with some automated translation, and cleared out from there.
More generally, as coldtea mentions, making integration into the rest of the language's ecosystem is the primary benefit of rewriting in another language.
The value of such a port to others depends on how easy it is to integrate between the two languages, either via libraries or other methods. The harder it is to integrate the two (and the absence of automated translation tools) increases the value of the rewrite to others.
Your Core-NLP example is actually an interesting one, because that library has already been ported to other languages... It is available for the C#/F# ecosystem (http://sergey-tihon.github.io/Stanford.NLP.NET/).
GitHub.com/sudhirj/restis
Also wondering if some rethinking is possible - would a HTTP interface a la DynamoDB be more useful? Can complexity and performance be increased by using a purely memory backend with no disk persistence? If there were pluggable back ends would a Postgres or DynamoDB back end be more useful for terabytes / petabytes of data? Is the beauty of Redis the API or the implementation?
The answer is "no" with a certain amount of probability. Redis isn't single threaded by lack of capability, but by design. Concurrency for multiple CPUs will actually slow down a lot of the stuff you see, as you will need to introduce locking mechanisms.
Also, garbage collection is highly tuned and customized in Redis to the use case of an in-memory-DB (in stark contrast to usual allocation patterns of an application), up to the point where it's almost impossible to replicate the performance in a garbage collected language.
I love Go and we're a 100% Go (and Angular) shop, but for an in-memory DB it wouldn't be a sane choice.
There should be minimal overhead from having the capability in Redis due to the way it implements disk snapshots (RDB snapshots are done by fork()'ing and relying on copy on write to let the main process keep on doing its thing while the child process writes the snapshot, so the main process doesn't need to care; other than that Redis offers logging/journalling of changes, but the cost of having that as an option is trivial if it's switched off).
Having pluggable backends for things like Postgres or DynamoDB seems a bit at odds with the purpose of Redis, which is exactly that you pay the (very low) cost of in-memory manipulation of simple data structures, though if a single Redis server could partition the key space between plugins, it might potentially be useful by letting you e.g. move keys between backends and still access them transparently to the client. E.g. for the samples I mentioned above, we roll up and transfer data to a CouchDB instance for archival now (doesn't matter that it's CouchDB really - we just need a persistent key-value store; Postgres or DynamoDB would also both have worked fine), but if I could archive them while still having them visible in a Redis-like server together with the in-memory keys, that'd make the client a tiny bit simpler.
For most Redis usage, I think paying the cost of connection setup and teardown and sending HTTP headers etc. would likely slow things down immensely. At least it would for my usage. Having a HTTP interface as an addition might be useful in some scenarios to enable new use cases, but as a replacement for the current Redis API would be a disaster.
If you want to explore alternative interfaces, I'd instead suggest going in the opposite direction, and experimenting with a UDP interface. In a typical data centre setting packet loss is low enough that while you'd need retry logic, it wouldn't necessarily get exercised much in normal situations.
(On the other hand, for the typical request/reply cycle it might very well not give any benefits vs. tcp in most scenarios where multiple request/replies are done over a single connection and thus amortising the connection setup cost - would be interesting to benchmark, though)
It's a feeling less of maintaining boxes, and more equivalent to paying a Redis-as-a-Service provider.
Looks like a really cool effort but authors of open source projects often think people would read the code and figure out all, the truth is people usually look at what's in the readme and that's all the attention span most people are going to have. My 2c: improve your README.md.
And if you read "Why? To learn rust" and ask "should I use this in production"...