Hacker News Reader

Comment by ccleve | Hacker News Reader

I have spent quite a lot of time building a database in Java. It's not a small undertaking.

I've gone through as many books and papers as I can, and there is only one, one, that gives an overview and actual examples of building a system that processes SQL. Everything else is theoretical or so general that you don't get enough direction to get started.

It's "Database Design and Implementation" by Sciore. It's a little dated, but not by much. It is an underappreciated resource.

View on HN

You may find MIT's Database Systems course helpful, the labs consist of building several db components in Java (and the prof is Stonebraker, of Postgres fame):

https://ocw.mit.edu/courses/electrical-engineering-and-compu...

https://youtu.be/F3XGUPll6Qs

http://blog.marcua.net/post/117671929

Stonebraker is the Donald Knuth of databases.

You could have read the source to the processor of one of the many open source SQL databases though (off the top of my head: MySQL, PostgreSQL, SQLite, Derby, H2, HSQLDB). Many of them have very clean code, SQLite would be my choice if you know C otherwise one of the java databases (though I've never actually read their source)

For some, reading a body of source code without any guidence about what path to walk or where to focus your attention is really quite a hard way to learn.

SQLite is quite easy to read; MySQL on the other hand I find very annoying. I used to run a very large (millions of accounts) shared hosting service for which I did a lot of MySQL customizing to remove limitations or remove features that would compromise our systems if abused etc and it was a nightmare to work with that codebase. I had to and learned a lot, but I would not do that again if possible.

It is an essential skill for our job though, being able to dive into a new code base, form a mental model of it, and modify it either because of new feature requirements or simply bug fixing.

And it is a skill, I also had a lot of trouble with it initially, but diving into several open source projects was the perfect way to practice it, and it helped me tremendously in my professional development.

Reading the source doesn't tell you why they chose that route or what the tradeoffs are.

Well, depends on the quality of the documentation I guess.

I agree with you, in that it's an essential skill for our job. However, from my experience at least, I've found that going head first into the source code of something you don't really understand how it works, is not the best course of action.

For example, I would make sure to read about B+ trees first, before reading the implementation of indexes in SQLite or PostgreSQL.

Agreed. A good tip for this is to try to find the earliest version/commit of the project you are interested into.

Most code bases are business logic and not nearly as complex or formal as a db. DBs, OSs and compilers are systems that should be studied to be effective in the code.

The thing about it that I find hard is knowing how to judge the important bits to read and knowing where to direct my attention as I read. Any tips on learning that?

I find that without knowing where to direct my attention, it is really hard to avoid getting distracted.

This.

Here's an anecdote: one my friends worked for a startup that at the height of NoSQL hype (circa 2012) decided to build their product on Cassandra. The data they dealt with was highly relational with hard requirements on consistency. He told me working on a NoSQL database taught him a lot about relational databases, because he scoured the Postgres source code to learn how to implement consistent and durable commits across distributed/sharded nodes.

OK I'll say it... Your friend chose wrong. HORRIBLY wrong. I have a lot of experience with both Postgres and Cassandra and the fact that your friend chose Cassandra despite their data model being relational AND having high consistency requirements shows that they chose Cassandra for the wrong reason(s).

Before you come back saying something about scalability, I've run both databases at way above average scale so what you choose should come down to what you need, which is not what was done in this case (your friend)

I think his comment was mostly about the fact that you can learn a lot about relational databases by reading Postgresql's source. He did say at the height of NoSQL hype. To me, that choice of words gives subtle hints as to the historical context and a disclaimer about the pertinence of that decision. Thank you for your input though, it provides food for thought for the next person mulling over a similar decision.

It sounds like his friend didn't choose anything, his bosses did.

>Your friend chose wrong. HORRIBLY wrong.

Yeah, obviously. Also, he didn't choose anything. The tech stack was mandated on him.

Sometimes the best way to learn the value of something is to try really hard not to use it.

Agree! I've accidentally found this gem. It is really written from the actual hands-on engineering/programming perspective which covers all aspects, from the front-end to the actual storage implementation.

Sciore is a great teacher! I had him at BC 20 years ago for both database systems and object oriented programming. I didn't know he put out a few books, I'll have to check them out.

I've been looking for this type of book. Thanks!

Reply on news.ycombinator.com