Show HN: Gun database – 100k ops/sec in IE6 on 2GB Atom CPU
github.com
github.com
- Redis is a server: all commands involve network or IPC round trips. It is meaningless to compare it to embedded data stores such as SQLite, Berkeley DB, Tokyo/Kyoto Cabinet, etc ... because the cost of most operations is primarily in network/protocol management."
That's not to say that the comparison here is bad or good. But it is valid given the diversity of use cases, the complexities of the technical differences you mention, and the pressure that blogs and other media place to adopt large scale solutions for small scale applications.
If the comparison were well handled, delving into the architecture tradeoffs, it could be fascinating. But this was just a cheap ploy to get clicks. Now the title has changed, my acidic tone is no longer justified.
For what it's worth, according to the guidelines for 'Show HN' an acidic tone is inappropriate. People expose themselves to significant psychological/social risk when posting their own work online as their own work. Show HN is intended to provide a safe productive space. In general, an acidic tone is a bit at odds with HN's general guideline of civility.
expose themselves to significant psychological/social risk
"Compare their string values with JSON.stringify, choosing the greater of the two."
In other words -- don't ever use this database in production.
on edit: Here's why. CouchDB, which is also an AP database, puts conflict resolution into the hands of the user, but each document contains a revision, including information for the user to determine which is the correct revision. Many times, it's just "the correct one is the most recent one", but that's done in the user's code.
Making this kind of assumption (stringify and string comparison) for conflict resolution is dangerous because there will be scenarios when you will lose data no matter how many times you post to the database, guaranteed. Yes you can't be consistent, but you need to tell the user that there's a data conflict and have them deal with the logic.
Yes, it is a very basic hybrid vector/lexical/timestamp CRDT. The naive-ness of it is also its power, because it has strong eventually consistent guarantees.
However, it is intended to be the base case. You can build your own custom CRDTs on top to handle more specific business logic. For instance adding a DAG, a ledger, a simple linked list, etc. gun's default algorithm is not a silver bullet but it does "just work" out of the box for many web cases. See more here: https://github.com/amark/gun/wiki/Conflict-Resolution-with-G... .
Please especially read about our CAP tradeoffs (see the sidebar on the wiki).
Finally, it should be noted that because gun uses a graph data structure, JSON.stringify only ever applies to atomic primitive values (not complex/compound objects). So it is safe to apply (it is NOT safe to apply to complex objects).
But you take time from the system. Your writes may converge, but to an nondeterministic state.
....I'll let everyone draw their own conclusions.
It looks like their benchmarks are particularly friendly to their cache size.
The biggest difference is that you still have to speak to Redis over the wire within a single machine, where Redis has it cached and replies. The (terrible terrible?) choice of JavaScript gets around this, once cached, it is directly in the app logic's memory - so you don't even have to perform any request. And that is where you get the speed gain.
Summary: GUN's caching is closer to the end result you are measuring, therefore it is faster. Not magical algorithms.
- What does GUN do badly?
Banking. Anything relating to strong consistency. GUN is an AP system which means you really shouldn't use it for anythign involving accounting, money, etc. - read more about that here: https://github.com/amark/gun/wiki/CAP-Theorem
Where should you use it instead of Redis? When you want realtime updates or graph data. There is a 10 minute guide on how to use graphs from a key/value, table or relational, or document oriented way here: https://github.com/amark/gun/wiki/graphs .
The P2P protocol's base case is an ad-hoc mesh network (how the messaging between peers works is explained here: https://github.com/amark/gun/wiki/Mesh-Network-Messaging-Alg... ).
Initial connections are handled statelessly, both peers exchange blobs of the requested/subscribed data to make sure they are in sync (not out of date / old / stale). On the wire, this is a GET command.
Then the connection becomes stateful, and only the changes/deltas/diff of data are pushed over the network, thus allowing for realtime updates. Over ther wire, this is a PUT command.
All data is represented as a graph (whether key/value, table, relational, document, or graph itself).
And that is about it! Other than, of course, the AP aspects and the hybrid vector/lexical/timestamp CRDT for conflict resolution. The architecture is genuinely quite simple, intended to be the foundation for more complex data structures to be built on top (other CRDTs, or DAGs, simple linked lists, etc.).
There is a half baked "How to port GUN" guide here: https://github.com/amark/gun/wiki/porting-gun .
Db made in nodejs? Check.
No mention of multiprocess? Check.
No mention of persistence? Check.(lol @ s3 persistence)
All tests failing ? Check.
Optimizing if/else? Check.
Faster than redis? Check.
Store non-optimized json in disk(not sure but looks liek it)? Check.
I cringe at every page on their docs(like sharding). It just feels like the templeos situation. Upvoted for discussion though.
Master/stable branch is not failing, and we definitely don't recommend using the JSON dump in production (we have warnings about this everywhere) it is intended to make local development easier.
NodeJS is single threaded, but because GUN has a P2P/decentralized architecture it can easily sync between machines or threads and handles concurrency and conflict resolution. Check out this demo - https://youtu.be/-i-11T5ZI9o .
How can you say "not sure" and "check" in the same line?
Huh? TempleOS is explicitly not meant to be one's primary OS or even really anything besides an educational environment. Gun appears to actually be trying to target production use.
If anything, this is closer to a "MongoDB is web scale" situation.
Can we please let HN be free of click bait? :(
(1) How many people on HN do you think are aware that they can get a huge performance boost by using an embedded cache rather than just relying on Redis or others? Personally I think there is merit to this point alone, most people overlook it.
(2) I linked to Redis' performance page, and even with pipelining turned on they are doing 0.5M reads/sec while gun (on the exact same type of test) is doing 25M ~ 30M on the same device (MacBook Air). So "50X" is a claim that can be backed by evidence, not link bait. If it is shocking to anybody, then that is actually why we need to talk about (1) more, to get the word/discussion out.
It's not a comment on your technology, architecture or approach.
But you're not sketching out a clear, 2 minute overview of what your approach is, with the pros and cons, and intended use cases. You're just throwing around random synthetic numbers, and apples to oranges comparisons. It's handwaving.
(A) There is a 2 minute summary at the bottom of the page, with the pro/cons/dangers of benchmarking. But that doesn't mean benchmarking should be ignored, it is a legitimate science.
(B) My previous comment's (2) addressed the apples/orange, it would be nice if you could explain why it is handwaving when I tried preempt that directly (to which you ignored?).
(C) There is a fair question in my previous comment's (1), which you also didn't address - would you mind answering it?
I love having conversations with people. It would be easier for me to understand what you are trying to say, though, without being accused (ad-hominem) of being handwaving. Thanks, I look forward to hearing your thoughts.
Cheers!
really?
"Compare to Redis at 0.5M ops/sec (cached reads, Macbook Air), even with pipeline optimizations turned on, here: https://redis.io/topics/benchmarks"
We also detached this subthread from https://news.ycombinator.com/item?id=13249586 and marked it off-topic.
I've lost friends to suicide secondary to PTSD... Maybe we don't overload that particular initialism...
And unless the dev is not a native English speaker, then I strongly suspect this was done as a little attempt at humor. For many people, it's just not funny.
I mean, absolutely use jokes like this if you want, it's your right. But then don't be surprised when people discuss the inappropriate terminology instead of your project's technical merits.
This past fall a friend and colleague suffering from PTSD took his own life with a gun... You can see why I feel the joke is in poor taste.
My condolences on your loss and grief.
(Not that you weren't!)
you're not even close to the reality of it, you might or might not even be aware of how heavily knocked you are, and it cold go on doe months or years
... "coming to" mid crossing a six lane street narrowly missed by a vehicle... how did you get there? will you reenber the incident two minutes later? or ever?
vr immersion and opiods play vital roles in creating non traumatised memory paths in your brain, and since opiods can be substituted for vr immersion for pain relief in multi amputees, to a considerable degree, that link is worth checking too.
That doesn't mean we shouldn't ever be serious, but I'm not convinced that our collective sense of humor should be modulated down to the least tolerant and most sensitive among us.
Dark humor is favorite pastime of folks working in fire/EMS/law enforcement. We are _very_ good at laughing at things that would probably be shocking to many. There is, however, a time and place for that. Making light of something like PTSD is not especially helpful to anyone.