Show HN: Knuckleball – An in-memory data structure server | Hacker News Reader

Show HN: Knuckleball – An in-memory data structure server

github.com

100 pointsrodrigoalima9936 comments

Amazing. Sometimes things are done for academic pursuit. If everybody would stop reinventing the wheel then we will never get any alternatives at all. The product does not need to have distinguishing features from the onset, it may gain it later in its lifetime.

Building a data structure server is great exercise in understanding a lot of different things: networking basics, efficient storage, fast retrieval and memory safety. Obviously I don't mean to claim that the author of the project necessarily had the same aims.

> If everybody would stop reinventing the wheel then we will never get any alternatives at all

Exactly. People complain about reinventing the wheel, but compare the wheels we have today with the wheels we had 2,000 years ago, plus all the different wheels in between.

The wheel has been reinvented many times throughout history, and we are all better off for it.

Indeed. Ever seen how much technology goes into a modern car wheel and tire?

Hi, thank you all for the feedback! I'll update the README file to make the use cases more clear. I know Redis and it is great! Knuckleball is a server that I started to develop a few weeks ago for learning and academic purposes, not to compete against Redis.

A data structure server is a nice and lofty goal for that I think. It has networking, command/protocol parsing, async operation, nice data-structure work so fairly well rounded project.

I don't think everyone was naysayer about your submission. Some including me were confused about the purpose since it reflects how we should look at it. If it was done to explore redis like server but with different priorities/knob everyone would have wanted to explore if that helps their use cases or how that affect the tool. Since its for learning the focus might be how readable/understandable the code is and maybe trying to learn from it themselves.

I hope you learn the kind of things you were planning to.

I've had written a lot of C++ code 10 years ago. Before that, I've spent a good deal of time writing C. Ever since, and even though I've started to do Go 4-5 years ago, I wanted to "get back on the horse" with the C++14/17 movement.

It's been a scary thought; I'm dreading to realize all the know-how's I once had known, that I've managed to forget. So in the meanwhile I've been looking at starter-projects, big C++ projects (somehow, there aren't small, simple ones) to understand how modern C++ projects should be _structured_ these days.

To those looking at this negatively I say - this projects is great for what I've been looking for. You can take it and build a differently flavored server from it, and still have modern tooling (CMake, GTest). It is simple enough.

I knew enough C++ to be dangerous around the turn of the millennium (college), but haven't written anything in it since. I've often wanted to pick it back up, but the thought of putting that much effort into something that requires so much more effort to use safely than the languages I currently use has always kept it as a wish rather than reality. At this point, I think I've given up on C++, and I'll use Rust when I finally get around to going low-level again. If I'm going to give up the ease of an interpreted language, I might as well put a little more effort in and learn something that provides a bit more assurance for the work required.

Then again, we'll see if I ever get around to it. It's not like I'm swimming in free time.

I was (and am still) in the same situation. I managed to go the Rust path, just to realize I don't trust it will be valuable for the very long term (5y+) at this point. It'll be funny to save this comment and come back to it 3 years into the future.

However C++14/17 is getting a ton of new features and a fresh effort to push it towards Rust's innovation. I wouldn't be surprised that if this energy continues, we'll see a lot of Rust in C++.

What is completely certain, that 3 years into the future, if I look at this very comment I'm writing now, C++ will still be as common and valuable. Rust - not sure.

I know I sound like a conservative dinosaur now (but trust me, I'm not), but this kind of thought process you get only after 20 years in the field, and I do apply it only for topics that deserve it (systems programming).

That seems an entirely sensible stance. For me, there's a few other things that make me really want to use from Rust to ease some of the pain points of writing a language that low (error/null handling, destructuring match), so I'll stick with Rust for now. Worst case, I may need to apply my Rust experience to C++ later, but with the new features you are referring to and I keep hearing about, those features seem to be increasingly like what I would be doing in Rust, so there should be a lot of overlap of concepts and idioms by then.

I think this could be used as a lightweight bloom filter. One direct use-case I see is to build a cache of keys from some _other_ database during initialization.

In cases where the read exceeds write, such as genomic data processing, this can provide additional performance gain, as opposed to querying the database.

While redis _can_ be used, I think that this having a much smaller codebase is advantageous, in case one wishes to implement additional domain-specific requirements. Nice work!

Not quite sure I understand the use case here, is meant to make updating a complex cache object easier?

same use case as redis.

I like it! Like others have said, doing something for the sake of learning (or even just for the heck of it) isn't a lost endeavor. I've been toying around with the idea of doing something similar, but .Net based, mostly for my own learning.

I have not written real C++ in a while, but this code is very nicely organized and written. Can't comment if its modern idiomatic C++. Its easy to read and make sense of what's happening. IMO, that is not easy to do. Congrats @OP!

What's a good use case for this platform?

Looks pretty cool! The world could use a competitor to Redis (which is also pretty cool).

It sounds like a redis competitor - but I don't see any distinguishing features.

And the readme doesn't mention redis at all - is it possible that the author is simply unaware of the existence of redis, and has simply reinvented it? Because that would be absolutely hilarious :p

It would also be a fun project, though. Redis is something that I really wish that I had had the opportunity and the vision to build myself. Classic data structures as a service; too cool for school.

Hilarious as long as you aren't the one that implemented it..

Erm, I wrote this because someone wrote something on the line of: 'wouldn't it be hilarious if this guy made this because he didn't know about Redis'

I either replied to the wrong comment or it got edited.

At first I thought there is a chance that someone thought about the advantage of the same thing. I have always thought that cache products that lets you build your custom distributed data structure would be a marvellous thing and while Redis does not provide the same feature it sort of allow one to build much more complex cache data structure using its primitive. But given that its implemented as a single threaded async server I have to assume its Redis inspired.

Now the curious question is whether it provides some different feature, its a learning exercise or something else.

Actually the latest Redis does allow you to build your own data structures using loadable modules, as announced at RedisCon 2016.

see http://antirez.com/news/106

The Smalltalk-like query language is rather different from Redis commands.

One distinguishing feature might be that the author understands the code and how it fits to their specific use case. As an alternative to trying to adapt and integrate a big complex piece of tooling into a localized application it may make some sense [or not].

To put it another way, tools like Redis are often designed and documented and optimized based on assumptions of large scale deployment and sophisticated engineering teams. While this doesn't preclude small scale and/or individual use, it still assumes a significant level of time commitment: for example, a reasonable understanding to achieve a 95% chance of successful deployment on a random application might require 8-developer-weeks of experience. On a team of four with an average of six years experience, there's a good chance that some of the necessary time commitment has already occurred. On a team of one there's going to be a deep dive even if the person already has half the required experience.

I mean suppose Redis does everything that is necessary except in terms of persistence model, or more likely it did everything fine until now when I realized I need a different persistence model. Sure it's a rabbit hole either way, and neither is necessarily better, but changing the persistence model of my code to suit my use case may be easier than kludging on or choosing among other people's kludges.

To be clear, I'm not arguing that the author's use case is a snowflake. What I am proposing is that the author's knowledge and skill set and time availability probably are, and writing a key value store may be more productive and efficient than figuring out how to adapt and apply Redis.

As a gentle reminder to the commenters in this thread, please keep in mind the Show HN guidelines (https://news.ycombinator.com/showhn.html):

In Comments

Be respectful. Anyone sharing work is making a contribution, however modest.

Ask questions out of curiosity. Don't cross-examine.

Instead of "you're doing it wrong", suggest alternatives. When someone is learning, help them learn more.

When something isn't good, you needn't pretend that it is. But don't be gratuitously negative.

I assume people removed overly negative comments (or they were removed for them?), as at this point there's not a lot of negativity at all, and the top two comments are both pointing out that the Show HN guidelines, which are predominantly about being civil and avoiding negativity. It makes for a somewhat surreal reading experience.

When I saw first read the comments this morning, the now-greyed out comments at the bottom of this thread were the only thing posted. 5 or 6 comments, all variations on the theme "why bother, lol".

Same. From my experience, I think that initial conditions on a thread matter. From what I saw, I was worried the thread would snowball in the negative direction.

Most of the current comments are not abiding by the Show HN comment guidelines.

  In Comments
  
  Be respectful. Anyone sharing work is making a contribution, however modest.
  
  Ask questions out of curiosity. Don't cross-examine.
  
  Instead of "you're doing it wrong", suggest alternatives. When someone is learning, help them learn more.
  
  When something isn't good, you needn't pretend that it is. But don't be gratuitously negative.

In addition, not all software on the internet is meant to replace other software, compete with other software, or even be deployed in a production environment. The internet is more than large enough for lots of wheels of all shapes and sizes. Contributing to existing projects is good and so is writing your own programs.

I understand the underlying sentiment people are getting at - this seems similar to Redis - but we should keep an open mind. Maybe he wanted to write C++ instead of C? Maybe he was trying out boost::asio? Maybe he just wanted to write some code?

Correct me if I am wrong: redis scalars are limited to strings and this seems to support integers and booleans. The query language is also different. So, I must conclude that something else is going on. Slashdot, is it you?.

Reply on news.ycombinator.com