Drop-In Replacement for Memcached | Hacker News Reader

Drop-In Replacement for Memcached

memc.rs

143 pointsmfrw87 comments

This seems to be a concurrent hashmap (dashmap[1]) with a server speaking the memcached protocol. I know both memcached and redis use sophisticated strategies to reduce allocation calls and memory fragmentation, but this seems to do nothing like that.

[1]: https://docs.rs/dashmap/latest/dashmap/

Memcache uses an outdated and slow hash and hashtable. This is using jemalloc and a concurrent hashtable. Seems to be miles better.

Not this again...

https://news.ycombinator.com/item?id=28833202

> Memcache uses an outdated and slow hash and hashtable.

No it doesn't. A cursory scan through the code will tell you as much.

Exactly why I told so. I even started to improve it some years ago, but got sidetracked. The only nice thing about the code are signals on htable updates. But a proper, fast concurrent hashtable would work wonders.

Honestly, still kind of impressive that an off-the-shelf concurrent hashmap can get within even a factor of two of the best-in-class hundreds-of-engineer-years piece of open-source software and all the fine-tuning that major enterprise users have contributed to it. Like, given how much infra budget goes into running big KV stores, I'd assume that memcached has been optimized to within an inch of its life across almost the entirety of its codebase, all the way up to the really hardcore stuff. Being able to just... grab a package and get within spitting distance of it is kind of amazing. IME when you do that you're lucky to get within two OOM.

I agree, but on the other hand you know the saying: "90% of the work takes 90% of the time, the 10% remaining takes the other 90% of the time."

How much effort would be required to get memc to equal or surpass memcached's performance? Would it require rewriting the datastructures from scratch? Fine-tuning or rewriting the tokio?

> factor of two of the best-in-class

Would be surprised if it didn't, and most of the time it should probably be even.

It's the edge cases that separates naive implementation from industrial usage. And off the shelf concurrent hashmaps are already pretty advanced.

> use sophisticated strategies to reduce allocation calls and memory fragmentation

Both of these are things a high quality allocator (jemalloc in this case) has to to optimize for as well (in this context "allocation calls" corresponds to syscalls). For example they also allocate different sizes from different buckets.

Applications do have some abilities an allocator does not have, for example:

* It can move allocations, since it can invalidate pointers. This allows defragmentation/compaction to happen

* It can try to predict how long lived an allocation will be

But it's not clear to me if any of these are important for performance.

memc has a measuring performance section: https://github.com/memc-rs/memc-rs#measuring-performance

As an arbitrary example, on my machine if I run memtier_benchmark like so:

    memtier_benchmark --port=11211 --protocol=memcache_binary \
      -t 4 --ratio=1:1 --pipeline=9 -c 16 \
      -d 100 --key-pattern=S:S --key-minimum=1 \
      --key-maximum=1000000 -n allkeys

I get the following results:

    memc: 63999483 ops, 587092 ops/sec, 51.39MB/sec, avg 0.82 msec latency
    memcache: 63999936 ops, 2391777 ops/sec, 209.34MB/sec, avg 0.24 msec latency

memtier_benchmark also prints out more detailed stats, but I couldn't get those to work for memc. Not sure what's going on there.

If we set the set/get ratio to 1:10 (maybe more realistic), they get much closer:

    memc: 6399480 ops, 639989 ops/sec, 28.45MB/sec, avg 0.67 msec latency
    memcache: 6399623 ops, 1291566 ops/sec, 57.41MB/sec, avg 0.40 msec latency

A wireshark capture shows that memcrsd is breaking up the response packets, not optimizing network, sending a lot more packets than memcached. With the 100 byte size, memcached is packing 10 PDUs per packet and memcrsd is doing 1.

Using memtier_benchmark --pipeline=1 with "memcached -t 1" and "memcrsd -r 1" shows almost identical results (141943 vs 136550) on my machine (keys 1 -> 100000). The network is where most of the CPU is spent in both memcached and memcrsd cases.

I'm slightly impressed with memcrsd/rust scaling, 32 runtimes with 16 memtier_benchark threads with pipeline=9 can produce 4 million ops/sec. They need to fix the networking parts.

I think most will use horizontal scaling anyway, and simply shard the memcached's and skip the threading concurrency. Multiple memcached's with client side sharding is hard to beat.

Hm, would be nice if they had a comparison. More than just "it's Rust".

Sometime back I did a Golang based memcache server implementation with BoltDB (experimented with Pebble as well). I dropped that implementation out of fear that nobody will be interested in using it, have same question here I guess. Why would I drop the existing battle tested solution for something that might still have non-memory bugs.

Cool, but what problem is this meant to solve? How does it compare to memcached in terms of performance?

I am not among the ones who would blindly say 'rewrite it in rust', but given what happened with log4j, I would much rather choose a safe implementation given the memory safety provided by rust.

This also does not mean rust might not have its fair share of bugs; IMHO the probability is a little less.

Edit: I mentioned the log4j RCE as it was new & do not mean to imply that it would not have happened if we had a rust implementation. As the comments have mentioned, it would be the same in any language IMO

> IMHO the probability is a little less.

Exactly, Rust simply eliminates a certain class of bugs. You can still make some really major mistakes in the absence of these bugs, e.g. "goto fail" could have easily been written in Rust.

Most certainly opening the door to a diffetent kind of bugs. This mentally of tying bugs to a language demonstrates a very poor understanding of the issues themselves, and if anything a large impacting bug demonstrates wide adoption of an individual piece, most probably choosen due to its stability or ease of use.

Remember heartbleed?...

> Most certainly opening the door to a diffetent kind of bugs.

No? There's no law of conservation of bug class or something, you can eliminate bug classes without introducing a new one.

> This mentally of tying bugs to a language demonstrates a very poor understanding of the issues themselves,

No it doesn't.

> and if anything a large impacting bug demonstrates wide adoption of an individual piece, most probably choosen due to its stability or ease of use.

I don't think this has held true historically, and log4j is a good example of this (though there are many others).

> Remember heartbleed?...

A weird point to bring up since heartbleed was an out of bounds memory read.

The "goto fail" bug isn't a great example; Rust actually does specifically guard against this by requiring curly braces around the body of "if".

Here's an explanation of "goto fail", which includes a code snippet.

https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-got...

If C conditionals required explicit braces around the body, the second fail would have been redundant code instead of a security vulnerability.

Additonally, Rust will warn you if you have "unreachable" code, such as all the code after the unconditional "goto fail". Though, I'm sure modern C compilers would also warn about this.

All that said, Rust definitely does have plenty of bug classes left for people to trip over. Integer under/overflows come to mind (though they're just logic errors instead of undefined behavior).

In the same class as the integer over/underflows, Rust also doesn't warn for throwing away information when casting. So (my_value as u16) might be a completely safe transformation from an unsigned one byte my_value to a u16, but it might be throwing away the top 16-bits of a 32-bit value and Rust doesn't think that's worth any additional caution.

Rust also hasn't boarded the "People can't remember precedence rules" train like some modern languages. It has fewer operators with unnecessary values, reducing some opportunities for mistakes like C's increment footgun, but it does still have precedence and so you can still forget the precedence rules which the compiler is going to apply to your code.

But there are a lot of things Rust does catch, and there are even more things that aren't caught by Rust the language per se but are caught in idiomatic Rust because the language affords better ways to do things. Locking is an example, idiomatic Rust locking means you can't make the lock A but change B mistake seen so often in real bugs, because idiomatic use of Rust's type system would put the variable B inside the lock for B, if you didn't take the lock you don't have the variable to change it. https://doc.rust-lang.org/std/sync/struct.Mutex.html

You can enable Clippy lints for `as` casting, and I think some of the common "throwing data away" ones are caught in the standard configuration or just pedantic. https://rust-lang.github.io/rust-clippy/master/index.html#as...

Another example would be bounds checking, yes it is much better than silenty corrupting memory, but dying with a panic might not be the best solution in some cases, specially if the application is controlling hardware.

Not only Rust, this applies to all safe languages, just crashing also opens the door to exploits and denial attacks.

If the bounds check fails at compile time then the program doesn't compile. This seems like it should be a reasonable ask for any programming language, but, apparently not.

And it probably applies to all general purpose languages but of course it doesn't apply to all the safe languages that aren't general purpose. You can't have bounds misses in WUFFS. When you try to touch array[x] in WUFFS, the compiler constrains the type of x to have a boundary limit according to the size of the array, reasoning that you can't very well have meant for it to be possible to overstep the bounds of the array at runtime. Trivial arithmetic mistakes you might find in unit testing or worse production with other languages are often compile time errors in WUFFS as a result.

Except bounds checking fail at runtime like 90% of the time.

Well, like I said, in WUFFS this literally can't happen, WUFFS doesn't have any concept of runtime bounds checking, likewise it has no runtime overflow/underflow checking, if the compiler can't see why what you're doing will obey the constraints you don't have a valid WUFFS program at all (well, library, WUFFS is a special-purpose language so you're not going to go around writing whole programs in WUFFS)

And it's still a pretty sad indictment when a compiler is looking at your access, can see it's outside the boundaries and doesn't reject the program because the language (e.g. C) wasn't defined with this obvious expectation.

For me it's in the same category as the (now fixed in an errata) behaviour where as I understand it C++ 20 std::format didn't consider it a compile time error if you wrote

  format("check X: {} Y: {} and Z: {}", x, z)

... even though that's obviously an error, and it's obvious you can detect it at compile time, the original specification said nah, this code just blows up at runtime, good luck with that.

How would you write random app X? That is my point, I don't care about special niche cases.

I think a lot of people would already consider the cases you've declared to be relevant - where dying due to bounds failure is unacceptable - to be niche cases which need to do their own work to stabilise and not act like this is everybody else's problem. It isn't, for example, as if explicitly checking bounds isn't possible in Rust if you want to do something other than give up.

For example I would consider it acceptable if feeding badly corrupted MP3s to the MP3-capable CD-radio makes it stop working... until I disconnect the power and restart it. Nobody dies, if it's very annoying I should remove the offending MP3 file. Whereas if it causes random mayhem that's a problem because now troubleshooting is impractical and most likely the apparent solution is to replace the entire device.

And remember that such "turn it off and on again" is considered acceptable even in some systems you probably deem safety critical, so long as it needn't happen when actively dangerous. For some years trains in the UK with a beacon-based safety system couldn't handle waking up next to a beacon, their firmware wanted to start by not seeing a beacon, then encounter beacons while travelling - so waking up next to one was confusing and they'd go into an error state. The fix? Mostly "don't park the train there when it's to be switched off", but when it happened inadvertently, they would move the train a few metres with its safety systems disabled, turn the power off, wait a few seconds, turn the power back on and no longer able to see a beacon it was now happy.

That is where I disagree, when you buy a pair of shoes that are somehow uncomfortable to wear, do you live with the pain or go to the shop to exchange them?

Exactly that is what we are missing in the software industry, the niche cases of high integrity computing should be a common practice, just like most people usually don't put up with broken stuff when given the option.

We need more liability not less.

When I buy new boots they're uncomfortable for at least a week or two. Replacing the boots wouldn't help at all, the boots adapt to your feet and your feet adapt to the boots, I don't have money to get tailor-made boots that would fit to the millimetre on day one if that's even a thing -- which I suspect in fact it is not (that is, people who could afford such boots don't actually spend hours every day walking and so probably wear shoes which are comfortable but less effective).

User experience matters, and actually just dropping dead in error cases makes for a better user model than many outcomes. The user has a model of what the machine does, which is always over-simplified and often plain wrong, but it's vital to try to get to a place where the user's model doesn't contradict how it's actually to be used. Undefined Behaviour is the enemy of this outcome. The user definitely does not have a correct model of what Undefined Behaviour will be, as we see many programmers don't either, whereas "If I do this particular dumb thing, it stops working and I need to unplug it and start over" fits in the user model relatively well.

Computers aren't special in this regard, watch "Sully" (Miracle on the Hudson), the cockpit dialog is roughly correct. What does their QRH tell them to do in the event of dual engine failure? Turn the engines off first and then try turning them back on again. Didn't work because their engines were trashed, but if even one engine had restarted, those QRH instructions mean there would be no film, just a scary incident some passengers tell stories about and an aviation safety file somewhere, the plane climbs (slowly) on one engine, they're still an emergency but they can put it back down on a runway at their leisure.

Some people miracously avoid death by not wearing a seat belt, yet the statistics prove wearing it is still a much better option.

You misunderstand the recent RCE in log4j. It has nothing to do with memory safety and can be implemented in Rust without any difficulty.

I did not mean that the RCE of log4j is a memory safety issue, its just that, I would prefer to choose something where the probability of me having to wake up at the middle of the night and fix is less. Honestly, I would care less if it were memory safety or whatever, given I had to wake up at midnight :)

Just because it is written in Rust, it does not mean that you will have less bugs or whatever to fix. I could easily say that if it is written in Ada or Nim, then you will have way less bugs, as those languages are way easier to understand, and they are safe & fast & easy. Nim is just as easy as Python, and it is safe if you want it to.

I completely agree. My choice of words was not not right.

>it does not mean that you will have less bugs or whatever to fix

But it does since you eliminate an entire class of bugs, and a class that's generally the source of most exploits out there (eg. heartbleed comes to mind as a similar level of exploit but was a result of memory errors).

JVM does the same with GC, and so do a lot of other languages, but when it comes to rewriting non-managed code Rust is a great choice over C in my book, I would feel better about introducing a Rust dependency over a C/C++ dependency just based on that.

As for ADA and Nim, if it's some black box docker service then sure, otherwise if I have to include it in my build pipeline that's a hard pass.

> But it does since you eliminate an entire class of bugs

As opposed to what language? C? x86 assembly? You may be right, but I was using Ada and Nim as an example, not C. I could have used D as well. Again, the fact that it is written in Rust and not Ada, Nim, or D does not mean that it has less bugs.

In fact, I would even argue that even if we do a C vs. Rust, it still does not follow that just because it is written in Rust and not C, it has less bugs. There are all sorts of other bugs that could outnumber the memory-related bugs you may have in C. Can you really not see this possibility?

>As opposed to what language? C? x86 assembly? You may be right, but I was using Ada and Nim as an example, not C. I could have used D as well. Again, the fact that it is written in Rust and not Ada, Nim, or D does not mean that it has less bugs.

Yes, C, the language memcached is written in. The fact that it's written in Rust means that it's more likely to work in the first place because more people are actually using that language in the real world vs. Ada, Nim and D.

>In fact, I would even argue that even if we do a C vs. Rust, it still does not follow that just because it is written in Rust and not C, it has less bugs. There are all sorts of other bugs that could outnumber the memory-related bugs you may have in C. Can you really not see this possibility?

I don't feel like just random googling for the sake of argument - it should be easy to find, there have been many analysis of this kind and pretty much any I've seen concludes the largest source of vulnerabilities have been memory errors (I remember FF, Linux, Microsoft, etc.). So yeah if you want to be pedantic for the sake of argument feel free to keep arguing about the semantics - real world experience motivated initial Rust development and it's gaining adoption in places like cloud provider infrastructure and drivers because people dealing with these issues know what their most common bugs are coming from.

Further it wasn't even a bug. It was the well defined behavior working as specified.

> and can be implemented in Rust without any difficulty.

Not really. You'd have to fight the language and pull in some crates, to be sure. There's no native object serialization in rust and it's not really a concept in the language or ecosystem - except perhaps in the future with wasm runtimes.

There are numerous details of log4j that, in general, just don't apply to rust code.

edit: Since perhaps people don't believe me...

Rust has no equivalent of JNDI, pickle, etc. There is no way to send rust 'code' except, perhaps, wasm. Certainly there's no well supported way with decades of history of being ingrained into libraries, or patterns that support it.

Loading and running a java class is only the most absurd exploitation possibility with log4j. Without that, you can still slurp any environment variables (or other process information) and send it to any host. And you can probably do yet other stuff too, it is quite flexible ...

> Loading and running a java class is only the most absurd exploitation possibility with log4j.

Uh, yeah, the RCE was very bad. And not a thing with Rust due to not having object serialization.

> you can still slurp any environment variables (or other process information) and send it to any host

Yeah, you can write a logger in rust that does really crazy things like adding environment variables through some sort of parsing of a language construct. That would still be a problem, but a lot less serious. It would probably also be really unidiomatic, but that's a separate story.

I think it would actually be pretty tricky to do that in Rust without realising what you'd done, as seemingly happened to log4j.

The root of the log4j problem is a failure to distinguish between log format strings and the formatted log string, beyond that what happens is that Java allows a lot of flexibility at runtime which simply doesn't exist in Rust. In Java it's fine to, at runtime pull in classes you've never used before and run them, you can't do that in Rust - the types only existed at compile time. Indeed this is a key feature of Rust, because it means wrapper types are free at runtime, invaluable to the embedded firmware community who love type checking and other safety features but can't afford runtime overhead.

> In Java it's fine to, at runtime pull in classes you've never used before and run them, you can't do that in Rust...

Of course you can. Rust would be a severely limited systems language if it couldn't support dynamic loading. E.g. see

- https://michael-f-bryan.github.io/rust-ffi-guide/dynamic_loa...

- https://crates.io/crates/libloading

Notice that no classes come in here, the Rust Plugin trait used in that first page actually exists in the Rust code at compile time, as expected. You get to pull in machine code, by explicitly telling Rust that's what you wanted to do, but you don't get to conjure any types at runtime, because exactly as I said they simply don't exist at runtime.

The Log4j "lookup" behaviour was, first of all, a really dumb design decision even in Java but it's just wrong to pretend that it would be equally easy for it to be so catastrophic in most languages, not just Rust, even in C++ I think you'd probably realise you'd screwed up while developing a way for the log framework to run arbitrary code.

This whole discussion seems kind of overblown. Every mainstream systems language is capable of having this kind of vulnerability, Rust included. But runtime pluggability is part of the culture of Java code (and some Python projects, and Ruby, &c) and isn't part of Rust or Go's culture. So every Java project is potentially exposed to this kind of thing, but only the plugin-styled Rust and Go programs are.

You're much less likely to see a deserialization bug in Rust or Go than in Java.

There is nothing intrinsic to Rust or Go that prevents deserialization bugs; they're just not encouraged by idiom like in Java.

> every Java project is potentially exposed to this kind of thing

Many Java projects, but not all. Class-loading permissions in a Java program are subject to the security-manager policy for the program; class loading can be moderated or even fully disabled (once the main program is loaded, of course).

The default is to allow anything to be loaded, however.

If log4j was implemented in Rust that feature would have never been implemented in the first place meaning th RCE wouldn't exist.

If Rust occupied the same market niche (better and easier than C++ for large companies!) as Java did in the early 2000s when such features were considered desirable by paying customers, it totally would have been implemented.

Rust is able to learn from the many mistakes that other languages before it have made. But pretending that this means Rust is inherently purer instead of just "has less baggage, but will soon implement its own", is naive.

The not yet rewritten in Rust problem.

you beat me to it!

I don't get it. Just starting with the three main benefits listed on the website, none of these are compelling...

> Safe: Implemented in Rust programming language and using Tokio runtime ensures memory and thread safety.

Does memcached have issues with memory and thread safety? I've never heard of them if it does.

> Fast: Can process thousands of requests with minimal overhead.

For a network cache you want _predictable_ performance more than "fast" performance, so that you're able to predict how a system will scale. "Fast" is among the top priorities for choosing a network cache implementation.

> Compatible: memc.rs is 100% compatible with memcached binary protocol. No need to rewrite your code. Just start a server and use your favorite language/framework.

What about tools for monitoring memcached in production? Does it support consistent hashing across a pool memcached servers?

Ultimately: why?

Well yes, memcached has apparently had its share of memory-corruption vulnerabilities. https://www.cvedetails.com/vulnerability-list/vendor_id-1299...

What can this do that memcached can't? We use memcached heavily in PROD but I would be hard pressed to replace it with an upstart competitor just because its is possible.

Memcache is a BSD licence.

This is AGPL or commercial.

The biggest difference I could find was not in the capability of it as a drop-in replacement but in the change of licence as any changes you might have to fit your needs now has different terms.

I guess the bet here is predicated on that... are companies running versions of memcached that are much improved and they're not sharing those improvements? Could this be a way to make those improvements public by this replacing memcached over time?

I'm not sure how they could do a commercial licence once they've accepted contributions, if they don't have a CLA granting the right over the contributions. But IANAL, etc.

They do indeed have a CLA: https://cla-assistant.io/memc-rs/memc-rs

Ah, awesome. I did not see this mentioned in the contributing.md file (it may have been there, I just didn't notice it).

> Could this be a way to make those improvements public by this replacing memcached over time?

Those who wish to share their contributions will, regardless of license. Those who do not, will stick to code with BSD-style licenses.

Doesn’t matter the license. If I modified this code and didn’t distribute the code, I have 0 obligation to publish/share my changes. These are copyright licenses and license agreements. Distribution is the trigger for copyright licenses.

(Assuming we’re talking about US law) Since the previous comment was talking about commercial use, there’s no entitlement to use someone else’s code for commercial purposes at all absent some sort of license agreement (or waiver, or fair use, or some other exception). If the license granting the right to use the code in the first place requires sharing changes, then someone using the code with modifications has to either share the changes or infringe copyright.

Standard dual license commercial and AGPL that most companies use. These are copyright licenses, Once you allow me to download the software, I can do anything I want to it but distribute. Licesense only invokes on the copy hence a copyright license. Copyright licenses can only grant rights not take them away, so they can't control how I use it privately.

Copyright law is concerned with the making of copies, not just with their distribution. Fair use covers making personal modifications to software for personal use, but does not cover modifications for commercial use.

fair use only covers distribution.

"Examples of fair use in United States copyright law include commentary, search engines, criticism, parody, news reporting, research, and scholarship. Fair use provides for the legal, unlicensed citation or incorporation of copyrighted material in another author's work under a four-factor test."

https://en.wikipedia.org/wiki/Fair_use

Copyleft which is the AGPL only cover distribution.

"Copyleft applies only when a person seeks to redistribute the program. Developers may make private modified versions with no obligation to divulge the modifications, as long as they do not distribute the modified software to anyone else. "

https://en.wikipedia.org/wiki/GNU_General_Public_License

Biggest value prop seems to be that it's a single binary built off cargo, and that it's rust based. I wouldn't be surprised to see “single binary” become a selling point for container based shops even if all else is equal.

Soon we'll see entire docker compose stack packed into single binary and everything will go back again

It's pretty convenient options for local dev. I am working on a project that has 7 services but they can be bundled into a single binary that has all 7 and NATS packaged in (plus you can spin up multiple instances of the services). Very convenient and builds in a few seconds.

And after a few years, "modular architecture with dynamically linked libraries! lean binaries!"

AFAIK, memcached's only mandatory external dependency is libevent. And you can statically link that if you want.

Unless you're doing something very special, I'd guess that most container based solution will just get the memcached Docker image on Docker Hub. There's very little to be gained by building your own memcached image, with a "single binary" alternative.

     s/Drop-In Replacement/Rust partial reimplementation/

Reading their github I was under the impression that it was an actual drop in replacement, their capability status passed all tests from memcapable. Am I missing something?

it wont perform as well, memcached does a lot of its own memory management. this defers all the storage to a library. the eviction policy is also not the same. random eviction might catch some people off guard if your expecting what memcached does. it doesnt seem to support the ascii protocol either. didnt see anything similar to disk based storage or persistent storage memcached has now

Does memc.rs have the ability to do replication (master/slave)?

It's great that they made it, it's always good to have alternatives, and when they are made this well, it is just good for the ecosystem.

Ok. Why?

Because Memcached wasn't written by the OP, but this library was.

RIP LiveJournal

Stop using RAM for caching, SSDs are just as fast now with LevelDB/RocksDB and can store a magnitude more data. Look into SSDB [1] instead of Memcached.

I know it's Redis, but it's basically the same as Memcached if you only use the key-value set/get commands.

See this blog post [2] and this Gophercon 2021 talk [3] for examples of it's use in production.

[1] https://github.com/ideawu/ssdb

[2] https://wakatime.com/blog/45-using-a-diskbased-redis-clone-t...

[3] https://www.youtube.com/watch?v=35eIxI_n5ZM&t=10533s

SSDB is faster even if data size is >> ram size?

Otherwise, you’re going to be adding equivalent memory to get to equivalent performance between memcached and ssdb.

I mean, hard no on both claims. A memory cache (even memcached, which might not be state of the art) stomps all over the performance claims of SSDB. Memcached can do ~250kqps on one core, roughly depending of exactly what kind of cores are in question, and can typically turn around a request in < 50µs. SSDB is only "just as fast" if the rest of your stack is so insensitive to performance that you can't measure an order of magnitude difference in service latency.

agree - most sstables use memory looking on disk if needed. ssds do not have the bandwidth or low latency RAM has apart from the additional I/O path to get it onto or out of disk. Now, not to say memory+ssd can't rock ... but ssd!=DDRAM.

> which might not be state of the art

Sorry this stuck out to me. what would be state of the art in the in memory kv store world then?

I don’t know I’m just keeping an open mind. I imagine that Amazon or Google or one of those probably has something with an RDMA interface that smokes memcached.

There are no latency numbers to be seen anywhere unless I missed them. If the difference in latency between RAM/disk isn't noticeable in these workloads because you have enough slack in the system, the distinction doesn't really matter, you can use whatever (case in point, the linked article uses cost-of-egress as the sole motivation, it doesn't even mention latency or QPS at all.)

Memcached also has a mechanism for storing values (but not keys) on disk for larger-than-memory datasets which works well on big parallel flash drives.

EDIT: The website seems down but I was able to find a graph on archive.org via https://web.archive.org/web/20210721201604/https://ssdb.io/ but without latency, only QPS in some simple benchmarks. It's not slow, but if you're pushing high QPS-per-core or low latency, there's no indication it will serve as well as alternatives, and that can have a bigger impact on ROI than just the cost of disk-vs-memory for your servers.

Just because it's on the internet it doesn't mean it's true.

You might be confused how a CPU, memory, and I/O are interconnected. First, for an application to access data in memory, all it need to do is issue a MOV with indirect addressing. It might trigger a minor page fault which gets handled quickly by the OS. For an application to read data from disk, it must submit a request to the disk controller through the OS. I am ignoring a lot of details but that's all you need to reconsider the information in the links you shared.

Keep in mind that reading from memory is in the nanoseconds, while reading from NVME is in microseconds.

Doesn't matter if either way, with network overhead from talking to your application from another machine, it's measured in milliseconds.

Many caches including Memcached support batch gets as well as pipelining, so network latency won’t dominate.

Of course it matters. You probably haven't written low latency server daemons. A busy memcached might be serving 20k to 100k requests a second, it's not serving 1 user only. 100k requests all being translated to memory vs disk reads will have a huge performance impact.

Reply on news.ycombinator.com