Comment by nostrademons | Hacker News Reader

Comment by nostrademons | Hacker News Reader

I think you're looking for a set of roughly-orthogonal concepts that together make up a "database", right? Basically, you're trying to understand the different dimensions under which database products might make different trade-offs, defining the design space? If so, take a look at these concepts:

Schema. This defines how the raw bytes of a data record are mapped into semantically-relevant pieces of data for your program. The relational model (behind SQL) is the most commonly used schema for commercial database products, but also check out things like XML databases, JSON data-stores, serialization formats like Protobuf or Thrift, etc.

https://en.wikipedia.org/wiki/Data_model

Indexing. This defines what data structures are used to quickly find records on disk. Almost all the major products use B-trees, but these are not the only options: you can also have hash indexes, sorted string tables, bitmaps, bloom filters, etc.

https://en.wikipedia.org/wiki/Database_index

Concurrency control. This defines how the database deals with multiple clients trying to modify the same piece of data at once. Options include row-level locks, table-level locks, MVCC, STM, CRDTs, etc.

https://en.wikipedia.org/wiki/Concurrency_control

The memory hierarchy. This defines where the data is stored, physically. Is it on disk, like most data-warehousing stores or distributed filesystems? Is it in memory, like memcached or Redis? Is it on Flash memory? Is there some sort of caching scheme where parts of the data are in memory and parts on disk?

https://en.wikipedia.org/wiki/Memory_hierarchy

Distribution. How is data split across multiple machines, and then how are failures in the network handled?

https://en.wikipedia.org/wiki/CAP_theorem

Hope this helps. Most data-management solutions (even ones that often aren't considered "databases", like memcached or Redis or flat CSV files or MapReduce or the Google Search indexes) can be mapped onto this space.

Yes, this is close to exactly what I was looking for. Between this and a database management systems book (as per paperwork's suggestion) I think I would get both "a lay of the land" and "a cathedral to build on" when it comes to understanding database systems. Thank you.

Old but classic is C.J.Date's "An Introduction to Database Systems". This book covers most concepts required for databases, and the reason why relational databases had been an improvement over prior key-value, hierarchical, and network databases.

One could consider relational language as a database kernel language.

OK, I thought relational modelling was more niche than that, thanks. But as long as we're on the topic of the kernel language model, I'm getting reminded of the fact that every time we add a stratification to the KL, we don't get a _better_ language; instead we get a _different_ language, with its own set of strengths of weaknesses.

For example, if the relational language is the database kernel language (model?), why does the "degradation" towards key-value pairs benefit a cache like Redis?

I'm all for principles books on databases right now, so thanks, I'll take a look.

EDIT: oh, CJ Date wrote "Database in Depth: Relational Theory for Practitioners", which seems newer. Do you know how they compare?

I just finished Database in Depth. :-) It is short and readable. It's targeted to working database programmers with a few years of SQL already. I learned a bit, but it was a bit of a bait-and-switch: the book is really a polemic about why SQL is not relational enough and people should use the author's own Tutorial D. Every page belabors that point. More examples are written in Tutorial D than SQL. I was hoping for some math & theory, and there was some theory, but there was little effort to connect it to SQL except to show SQL's shortcomings. For instance I never once saw the term "outer join" (because Date wants to forbid NULLs). If you are interested in database theory and trying new things, it might be very interesting. If you are a working programmer who wants to solidify the foundations, there are probably better choices (I hope).

On the kernel language topic, though, Date and Darwen's work [0] on the specifications for an ideal class of database languages unifying OO and Relational models (D; of which Tutorial D is one realization intended principally for pedagogical purposes, hence the name) and their lower level work on an abstract relational algebra (A) to underlie D is probably useful in that regard, as frustrating as some of their work may be to people who just want to get down to using current SQL-based relational DBs.

[0] available at http://www.thethirdmanifesto.com/

Reply on news.ycombinator.com