Hacker News Reader

Comment by jdw64 | Hacker News Reader

Is it really the right choice to drop Redis and go back to a disk based relational database just to wrap transactions into a single unit?

Redis handles tens of thousands of concurrent connections in a single event loop, while MySQL uses one thread per connection. No matter how I look at it, that seems like a step backward.

Of course, performance isn't everything. And if performance isn't a problem, having everything in one place does make it easier to reason about. But I'm worried that under spike traffic, this approach might actually cause more problems.

I think putting a scheduling layer in front of the DB would be a better approach. The application server could handle concurrent connections and only write to MySQL when correctness is actually needed. That seems like a cheaper way to do it. but is it different for large-scale enterprise distributed systems?

View on HN

Redis doesn't have transactions and persistence.

No persistence means the data gets lost if machine shuts down or process crashes. Furthermore, after restart you will need to regenerate the data which can take time. That's why Redis is a cache and not a database. You can fix the persistence issue (Redis can write WAL log, don't remember if it does fsync or not), but then Redis won't be able to handle those thousands of concurrent connections.

Redis (and other NoSQL storages) don't have some magic architecture that gives them advantages over SQL databases. They just cut corners on ACID guarantees and skip fsync. Once you start doing fsync, your transaction throughput will drop to SQL database level.

Redis also doesn't have transactions which means every app error damages the data. You will spend engineer hours investigating and fixing the problems. Transactions save so much time and worries.

Does Redis become that slow when you enable both AOF and RDB? Sure, there's a write cost, but it doesn't lose its ability to maintain tens of thousands of connections. Redis supports AOF and lets you choose the fsync policy.

But I think using only MySQL is unnecessarily expensive, just to get single transaction tracking for bug tracing. So the article's argument seems to be:

'Use only MySQL as a solution to the distributed transaction consistency problem between two different storage systems, Redis and MySQL!'

But I think using Redis is much more elegant. It's easier to scale. I'd even argue that something like Saga would be a better approach. Of course, we might just have different opinions. But in my experience, reducing layers always ends up making things more complicated in the long run.

p.s. We have different views, but I do think some of your points are valid, so I upvoted your comment

The fsync policy equivalent to SQL database would be "fsync on each change before reporting successful update to the app". Redis (as many NoSQL databases) also doesn't have SQL and is a pain to view the data, you need to write extra tools when investigating the problems.

RDB snapshots can cause multiple page faults due to use of fork() and CoW.

> It's easier to scale

The company in question manages online stores and they could easily scale by allocating a separate database for each store (sharding).

> But I think using Redis is much more elegant.

I cannot agree because I think using a single database for all the data is more elegant, than multiple different databases and there are less problems to deal with. I dislike microservice-style architecture strongly and believe it is mostly good for wasting company's money.

> 'Use only MySQL as a solution to the distributed transaction consistency problem between two different storage systems, Redis and MySQL!'

I read it as "do not create unnecessary work by using a single database".

It's interesting that our views differ. I think it's because of our different experiences. I believe MSA is the right approach. But this doesn't seem like a debate that can be resolved through discussion. It's something that needs to be implemented and tested.

Still, I respect your perspective and your experience. We clearly have different values, but I think you have a mature engineering mindset. Ultimately, I think only real measurements can settle this. Have a great day.

I became curious how fast fdatasync (a better version of fsync) is, and asked LLM to generate a microbenchmark for me. The benchamrk accepts records size, number of records, opens a file, and appends records one by one, making fdatasync after each one. On my SSD, the throughput is 120 fsyncs/second, or ~0.5 MB/s. So if I had Redis to flush the log after each operation, it wouldn't be able to serve those thousands of connections. That's why Redis is either used without AOF, or they flush the data once per second. And that's why SQL databases might look "slow".

Of course one could optimize this - for example, while fsync is being executed, we could accept the queries from other clients and execute them, and once previous flush finishes, flush multiple transactions at once. However, I am not sure if Redis can do this due to being single-thread.

And obviously maybe there are problems with drivers, or with my consumer-level SSD and maybe "professional" SSDs can do more flushes per second.

Writing the code took less than a minute. I often do microbenchmarks now because it is so easy.

Concurrent connection count and durable write throughput are different things. And AOF doesn't necessarily mean one fsync per record. The benchmark is taking the worst-case speed of Redis and comparing it to a best-case SQL benchmark.

Redis AOF isn't one fsync per record—Redis can use group commit to share fsync across multiple commands. In other words, that could mean a difference of tens of times in your benchmark. I see it differently.

my common dumb test approach with fio

  fio --rw=write --ioengine=sync --fdatasync=1 --directory=test-data --size=220m --bs=2300 --name=mytest

Consumer grade SSDs 100-500 iops, datacenter SSDs in thousands (like ~ 5 year old samsung model on Hetzher AFAIR had ~ 3000 iops in that test). Cheap VPSes - easily in 5-30 iops of this sort

more often then not, fsync/fdatasync is overlooked by programmers

Everything you said is incorrect. Redis does have transactions, as well as data persistence. All cloud providers provide managed redis instances with automatic backups as well.

I mentioned that Redis can write changelog (called AOF in the docs [1]). However, if you tell it to do fsync on every update (like SQL databases do), it stops being that fast and spends time waiting for the filesystem.

Furthermore, the RDB snapshot mechanism (when Redis forks and forked process writes the snapshot) can double memory consumption and cause thousands of page faults in Redis process if there are many writes happening.

The docs contains corresponding warnings. "Cloud backups" are marketing terms and not ACID guarantees.

As one more disadvantage, Redis has no SQL and you cannot easily view the data.

As for transactions, indeed it seems to have them, but their execution is serialized, i.e. when MySQL can prepare 100 transactions in parallel, Redis will execute them sequentially.

[1] https://redis.io/docs/latest/operate/oss_and_stack/managemen...

Reply on news.ycombinator.com