Hacker News Reader

Comment by jensenbox | Hacker News Reader

I would imagine that this project was a Not Invented Here sort of thing when Redis was presented as an option.

Total conjecture on my part.

This is based on RocksDB which is "sorted key value" store like LevelDB (HBase, Hypertable etc.) and keep sorting and merging keys while flushing to disk at certain point.

Redis in comparison is a different thing if I'm not wrong.

It's a valid point - RocksDB can be thought of as the internal backend implementation of the DB. It's the front end API that matters as to whether Redis could fulfill the same purpose.

Why do you need junk then if rocksdb exists?

Juno is a disk-based store - a closer comparison would be to Mongo. And back when Juno development started, Mongo would not have seemed like a very good option (if it even is today.)

Also, Redis was not originally distributed.

> Juno is a disk-based store

The question is definitely interesting. And to be fair, Juno was originally in memory.

MongoDB is a document database not a key-value store.

That distinction is massively important when you're talking about distributed systems as key-value has far less edge cases to consider. Also its architecture is quite different as it doesn't have the concept of proxies.

Basically in the realm of databases the two are nothing alike.

You can think of key / value as a subset of document namely one with a single index

Ah yes, redis, the first distributed KV store that wasn't NIH.

Right and don’t you understand that Redis has the availability guarantees, scaling constraints, and memory architecture for all use cases?! Why would you ever need a different KV store when you have Redis!

> Why would you ever need a different KV store when you have Redis!

If you want to have tier based storage, where you trade off latency for increased data size, meaning disk space is your limit and not your RAM.

I was being sarcastic.

Everything in Redis has to fit in RAM.

My statement was sarcastic.

Is Redis safe? I make a SET request and before the data is written to disk, if Redis crashes, my data is gone for good, right?

edit: Redis supports fsync at every query in AOF mode

https://redis.io/docs/management/persistence/#ok-so-what-sho...

I was being sarcastic.

Redis was not a distributed system a decade ago.

KV database is just fun enough project to be NIHed many many times

That's how I feel about all of these such things developed by big companies.

If you're smarter than the hundreds of engineers designing and building the distributed systems powering the world's largest applications then well done.

Personally, whenever I see a system like this I try to look for why they didn't go with an existing solution. And in 99% of cases either an existing solution never existed or they had a unique requirement that necessitated building something from scratch.

Honestly, I work at a FAANG and, sadly, the answer is often a combination of people paying absolutely zero attention to tech that's NIH (e.g. SWEs working on DBs who have never used Postgres) and the fact that no one gets promoted for implementing a solution using 3rd party software. The system is setup such that you need to show you've developed something of sufficient complexity, and using OSS just doesn't look good in that context.

> DBs who have never used Postgres

PostgreSQL is great for single instances but is poor when it comes to high availability and horizontal scalability.

Be curious what use cases where engineers are writing their own database but it is single instance.

I'd expect an engineer designing a new database, even for internal use, to be familiar with the competition's strengths/weaknesses even if they're going in a different direction.

>If you're smarter than the hundreds of engineers designing and building the distributed systems powering the world's largest applications then well done.

There is plenty of absolute code abominations powering "world's largest applications"

Engineer competence is also vaguely related to quality of the infrastructure, yes, you need smart people to make big complex things, but you also need smart people to manage ungodly legacy enterprise spaghetti

You're assuming a level of engineering/scientific "purity" that doesn't really exist within orgs to the idealistic extent that we would like to imagine. I'm not saying that I'm smarter than every FAANG engineer, but having met quite a few and worked in large orgs myself, it's easy to say some of these tools come from pride/arrogance/a need to have your name on something/a fundamental misunderstanding of existing tooling. Not necessarily because they're unraveling parts of the universe yet unexplored and need a bespoke weapon to tackle new issues.

Reply on news.ycombinator.com