Sorting in key-value data model | Hacker News Reader

Sorting in key-value data model

antirez.com

19 pointsantirez9 comments

I have an idea about (more) persistent storage. I am not sure this is technically possible but anyway. How about setting up a cron job and running a backup script which get and save to disk the live content of the db at that moment? The format of this backup file may be CSV, SQL or native Redis one. What do you say?

He fails to mention that TokyoCabinet already supports ordering of the kv pairs through a function you provide to compare two keys.

If you want to iterate over all foos ordered by some property they have you just structure the keys so they are ordered properly and then jump to the first value.

The approach of using an embedded interpreter to perform sorting is absolutely viable and sane. With Redis my aim is to build a data model and a set of operations that do not depend on an embedded interpreter.

if you compare Redis to Tokyo Cabinet what are the advantages/disadvantages? One may be TC is stress tested by many.

docs are a great start but i hope antirez will extend the horizontal scaling/consistent hashing topic a little bit more.

one more thing example code with short explantions is much better.

Kudos for PHP support!

Hello, an early adopter, contributor, and well known hacker replied to your question better that I'm able to do I think: http://news.ycombinator.com/item?id=511978

I hope Python/PHP/Erlang bindings will follow the example of the Ruby bindings implementing consistent hashing ASAP but probably there is to take into account this 'hashing tag' thing discussed in the sorting article.

This is all great and stuff and makes you wonder what is up next. Maybe support for different data-types, maybe putting relations between data. Heck maybe even aggregating stuff stored in there!

Colour me untrendy, but I simply don't see the fascination or the need for key-value databases and I fail to see how any of them are not reinventing wheels invented 30 years ago.

Here's a key-value database for all of you, which supports sorting, aggregation, transactions and not to mention replication out of the box:

    CREATE TABLE KVDB (
       key   varchar(5000) PRIMARY KEY NOT NULL,
       value varchar(MAX)
    )

Feel free to use it.

Maybe I just don't get it, but deep inside I get the feeling this whole key-value database hype, just like sharding, is driven by MySQL developers refusing to give up on their sub-par database and move onto something proper.

having done mobile engineering, i can tell you that storing the data in key/value pairs is much easier and elegant solution then using a database, where you don't need the power of relational data. Both android, iphone, and windows mobile, provide some kind of database management (sql lite), which I think tends to be an overkill.

Doing SQL queries, and storing/retrieving data from a relational database is like an mental interrupt. You have to switch from object thinking, to relational, plus all the little details that you have to handle by using sql conecctions, it becomes almost too much work.

By the time you get to start doing queiries, i am done writing the app with key/value pair already.

Now, you can say that the server side is different than a mobile client. Sure, for many large websites relational data seems to be a must, but for small sites, they really don't look that much different form mobile client. key/value pairs might be just good enough.

And you just can't beat the simplicity of key/value pair systems. just as simple like using a hashtable.

I think there is nothing wrong with striving for simplicity. Plus with databases you have to deal with pool connections, prepared statements (to prevent SQL injections attacks), and much more.

Here are two pseudo-examples.

PreparedStatment statement = new PreparedStatment ("SELECT * FROM CUTSTOMERS_TABLE WHERE CUSTOMER.ID = ? ") statment.replaceValues("john"); ResultSet rs SQLQuery.execute();

while (rs.hasNext() { get your data here }

With a key value pair.

Customer = (Customer)CustomerData.get("John");

I agree in most of your points, but actually the greatest benefit may be for large systems, where performance is very important. It is strange to compare relational DBs and key-value DBs, they are very different bests.

I imagine key-value DBs, in the short term future, to be used like you use the memory itself in a program, with lists, hashes. Just this memory with your data structures is shared, atomic, persistent.

When we write code it is obvious when we put data in memory to use the most sensible data structure for the work, right? Incredibly when data is put inside a relational DB this is no longer true, and we create an absurd data model even if our need is to put data and get this data back in the same order we put this data (an ORDER BY is required when the data should be already sorted. Strange, dont' you think?).

Key-value DBs bring this back at home, to create sensible data models for the problem we are trying to solve.

Sorting a very central topic in computer science, I think it is important to explore it in the point of view of key value stores without the need of reinventing relational dbs.

Reply on news.ycombinator.com