A one-size-fits-all database doesn't fit anyone
allthingsdistributed.com
allthingsdistributed.com
As it stands, this post is a brief comparison sheet of AWS's datastore offerings, with bite-sized anecdotes that hint at a valuable usecase. It doesn't make a particularly strong case for the necessities, justifications, and tradeoffs of each particular type of paradigm or concrete offering, but is instead a brief content marketing piece with just enough practicality to justify its existence and make its point.
Worse, we actually declare the good parts (say, ACID transactions) of the old tech unnecessary and even dub it a feature that they are absent in the new tech.
Finally, the grace period expires and everyone seems to get the memo at the same time. The response is then "This is nuts. What the hell are we doing?"
The old tech is then restored to its rightful place and it's on to the next tech.
SQL Bad! SQL Slow! SQL terrible overhead!
SQL is just an abstraction layer. What makes relational databases "slow" is actually the overhead to implement ACID and implictly guarantee consistency and integrity at all times.
When Facbook losses a dozen status updates that will piss off a few users. When a bank loses a single transaction and depending on the value and importance of that transaction (I.e. mainting a position in FX, with deals that can go into the billions) this can literally kill the bank.
Besides: relational databases (which I'd wager are still the backbone of > 90% of businesses) are definitly not slow. And very few companies have such insane requirements like Google, or Facebook.
Edit: Wrong word that didn't make sense
What makes things slow with interactive transactions in distributed environments is explicit tradeoff of latency and availability for consistency. It just can't be done in bounded time [1]. But that's not the reason traditional databases are slow, they do not actually make this tradeoff in most deployments and are assuming you are ok losing some consistency due to network problems (yeah, so much for that ACID guarantees, you really need distributed algorithms to guarantee consistency if you talk to a database over an asynchronous network, like ethernet).
They are slow because for performance SQL is just a bad leaky abstraction and arguing that it can be fast, is like arguing for a sufficiently smart compiler [2] that can turn your high level abstraction into the fastest possible opcode.
[1] https://groups.csail.mit.edu/tds/papers/Lynch/jacm85.pdf
I prefer to stick with one single reference data store, be it MySQL, Postgres, SQL Server or whatever. If we need to add some kind of search optimised database or big data analysis database they should replicate from the reference data store and be treated as non-mission critical add-ons.
That is not likely to give you the same pace of development as a competitor who chooses to use a graph DB natively while your team goes through a labor-intensive translation or impedance-matching process for each change. If they're able to outpace you in development/innovation, you are more likely to lose in the market.
I find running a spark job on the 2 columns needed to build the graph a much better usage of time.
In my view, the reason why so many people are returning to relational DBMS for the primary data store is that data tends to live longer than you think and serve more than one purpose over time.
Choosing the "right tool for the job" is easier said than done if the job keeps changing.
These things always depend on business requirements. What relational database offers, above all others, is data consistency: if used properly, it's an incredibly useful safety net that prevents you from errors that corrupt user's data. Transactions and especially proper schema can catch a lot of bugs that could otherwise wreak havock.
So, it's a trade-off: how bad, from a business perspective, would some data corruption be? If you're Tinder, mixing up profile information or deleting a match is probably not that bad. If you're dealing with other people's money, on the other hand, it could destroy your entire business.
Also relation databases enforce their version of consistency. What if I only want consistency within one customer's data but not all customers?
I could have two inserts for operations on two different customers fail because of a deadlock on page slips in a secondary index. I didn't need that consistency but the database didn't know that.
I always hit come sort of limitation and come back to Postgres.
One day I'll just stop betting against Postgres, but I really do want to give some of those graph databases a try just for fun.
It baffles me a bit that CouchDB isn't more popular and that people rather use MongoDB for NoSQL, which doesn't come with the same syncing capabilities.
Agreed, it should be.
> and that people rather use MongoDB for NoSQL
I don't see why people would rather use MongoDB for anything, but that's just me.
> which doesn't come with the same syncing capabilities.
This is the main reason. CouchDB is amazing if you need to sync, but, if you don't, you may be better off using something else (like Postgres).
How does CouchDB handle conflicts, by the way?
Some hilarious bugs included a "relationships" collection that stored a record's identifier and an array of all the other documents it was related to. It was intended to be used for "joins" (yeah, I know). As the system grew and some customers created tens of thousands of records, all of which belonged to them so were represented in this collection. Eventually, the max array size in MongoDB was hit, which failed silently, rendering all of the data worthless.
I spent much of my time creating a roadmap for them to migrate to PostgreSQL.
I'm not blaming MongoDB here, it was just the wrong tool for that job. But some companies ate up through marketing material and thought it was the solution to all their problems.
I think that the original thinking was along the lines "well, we do not know what structure the data will have yet"
I am not in any way related to mongodb, nor do I really like it.
That being said, I can't image they failed silently. Much more likely that some return code from an insert was ignored in the application.
Otherwise that would be just the mother of all errors of DB development.
However, I do recall many questionable things like:
- Some sort of fast but in the worst-case inconsistent default config
- In order to execute the command to free space (which you had to do regularly) you'd need to have lots of free space
Both issues are probably 5 years old, so I also don't remember any details. Migrated to postgres a year later or so.
Failing silently is almost never the right thing to do.
That's the deadly combination.
If it was MongoDB + experienced developers who picked it out of careful consideration things would be different. But you never see that because experienced developers are smart enough to pick Postgres instead.
Seems like through every technology hype cycle, the same reminders need to be written and re-circulated again.
Stonebraker’s own Postgres does all of that.
The article is just an infomercial for Amazon’s (expensive, proprietary) offerings.
I agree. But we are still quite a young industry, certainly as far as relatively mature tooling goes. Perhaps one day soon the technology cycle will be taught to programmers in the same way that MBAs learn about the business cycle.
And followed up about multi-cloud here: https://blog.fauna.com/survive-cloud-vendor-crashes-with-net...
The relational model is incredible powerfull. Maybe graphs need something seriously specializes (and indexes, of course) but the relational model do fine:
- Relational - Key-value - Document - In-memory - Search
In fact, a index (that is the thing that need to be fit for the case) can be modeled as key-value, so, as relational.
Relational won agains the NoSql in the past because is MORE powerfull, flexible.
---
What need to change, is the specifics of the implementations. And yet, relational adapt fairly easy.
In fact, just look how many stores add a sql-layer on top. Ironically, SQL is what need a easy makeovers, no the relational model....
https://theburningmonk.com/2015/04/modelling-game-economy-wi...
I read this article I found on HN a few years ago, and for some reason it came to my mind just now.
The neat thing about the graph model of course is being more immediately expressive to the query-er, and that takes cognitive load away from that person.
I'll never say a bad thing about SQL or the relational model. I completely agree that it's superior in terms of flexibility, performance (almost always), modelling (usually), etc but I guess there are times where you have a weird project and it's just I guess the convenience and ease of using Neo4j and a few cypher queries and then it's over.
In the case of the article I linked, there is also the benefit of having a convenient way to see how changes echo throughout the graph, something that is more involved to do with tables and joins and all that. I'm not a DBA, I've really only ever dabbled, but if I got dropped into that situation and DIDN'T have a graph model, I'd have no clue where to even start. Maybe I'd start trying to walk tables and build graphs out from each item type to each other item type. That'd be gross.
Of course, that's probably not a problem scenario that comes up very often. Most of the times I've interacted with databases in general I've been just dealing with sanitizing params, getting HTTPS configurations configurated, just boring infosec stuff. That problem is probably very rare in the realm of "problems that graph seems to have an advantage"
Anyways, yeah. Relational, awesome. Graphs, sometimes convenient.
If your scale is small enough, fiddling with specialized solutions is just premature optimization.
PostgreSQL will take you a long way before you need to look somewhere else.
My comment doesn't necessarily contradict the article. I only mean to point out that a general purpose "non-optimized" thing still fit a whole lot of companies out there, depending on their scale.
EDIT: Although I concede that yeah some DB tech makes it easier to configure HA systems, if you can live with the downsides.
We see long articles about this very often here. Sometimes it's moving from relational to non, sometimes it's moving back. But what really throws me for a loop is that so many of these sites are so mind-bogglingly, face-punchingly slow. All of Atlassian's web properties are stupidly slow. AirBNB (mentioned in this particular article) also--painfully slow. Github--slow. Reddit--somewhat slow almost always. Twitter--slow. Facebook is usually the exception.
I realize that sounds like maybe I'm just on a slow connection, but I'm not. I don't know where the time is being eaten up if it's not in the database. But I feel bad for all the people working on db performance when the end result is so bad.
I don't understand how websites whose only purpose in life is to serve text and images as fast as possible can be so slow and that the companies that make them can find this acceptable. No database technology in the world can make up for bad product decisions and companies that don't consider speed a feature.
(1) Start with a relational database.
(2) Build product until you have a deep understanding of your market and their needs.
(3) Move to a specialized data store.
For established products, or well-researched use-cases, sure, pick a specialized data store. It will probably serve your needs better.
If you don't yet have product-market fit, use a relational database. It will give you far greater flexibility when you discover that "what we thought the market wanted" is different from "what actually made the right numbers go in the right direction".
> Ad hominem (Latin for "to the man" or "to the person"), short for argumentum ad hominem, is a fallacious argumentative strategy whereby genuine discussion of the topic at hand is avoided by instead attacking the character, motive, or other attribute of the person making the argument, or persons associated with the argument, rather than attacking the substance of the argument itself.
I might have added "for me" at the very end, so it would not sound general and was only describing my personal vibe. If this was a very interesting article about the importance of hydration and was full of references to coca cola products, I would feel the same.