Hacker News Reader

Comment by billdybas | Hacker News Reader

> The lesson we can learn from this story is the following: start with a generic database...SQL database are a good choice because they can do many tricks...The modern and successful architecture that is commonly used today is to have an SQL database that is sometimes surrounded by some one-trick ponies to take care of a few pain points.

Yup.

I like that we now have more of these one-trick ponies to choose from our toolbox when necessary, when a relational database just won't cut it.

But, my biggest complaint around the NoSQL movement is the marketing pseudo-hype it created. So many amateurs who don't understand database selection took it as gospel and evangelized it across the web (eg. Mongo w/ Node).

It's hard to correct people's understanding when they learn things wrong the first time, especially when there's a mountain of incorrect information they can point to on the web ("These people can't all be wrong, can they?" Well...).

View on HN

The quoted statement is not correct at least as far as analytics are concerned. Two examples:

1.) Analytics in many enterprises increasingly feed off data lakes consisting of enormous quantities of data in object storage. SQL has a part to play but it's effectively computing aggregates and creating data marts off this deeper pool of data. Data lake architecture is likely to be increasingly dominant given the enormous growth in data volumes.

2.) Machine learning is transforming analytics. This looks like the next feature likely to be absorbed into DBMS systems. SQL integration with ML is likely to be a hot topic in future systems but a substantial fraction of ML processing will remain outside the DBMS.

So SQL is going to be present widely in most future solutions but that's not the same as saying that a single relational DBMS architecture will solve all problems. It's been clear for years that ACID-compliant RDBMS have a part in this picture but it's just part.

Overall the article still seems to be fighting the SQL/NoSQL wars of the last decade. A large part of the market is moving on to other use cases.

Oh, yes, definitely.

Data warehousing and ML have different requirements and needs than your typical N-tier web app. Even streaming event data warrants a different solution. It comes back to knowing how to choose the right database for the job.

The issue is around how these technologies are marketed – grandiose claims and few practical use cases. Once the marketing material permeates the industry and some part latches on, it becomes a self-reinforcing cycle. Blog posts, books, and courses bring the information to the masses. Then companies start to adopt the tools. Then they need to hire engineers who know those tools. So more information gets published about them because that's what people want to learn to get hired.

Many engineers today will turn to NoSQL for everything because of the past few years of marketing hype (and acronym-driven-development), and that's quite a shame.

For those looking to understand how to choose the right database for the job, I'd recommend first reading "Designing Data-Intensive Applications" (https://dataintensive.net)

The most infuriating part of "NoSQL" for me was always the conflation of SQL and RDBMS and ACID. Most NoSQL is simply non-relational or non-ACID. You still need an access layer.

After all these years people seem to have finally realized that the challenge was never SQL, it was data and you still have to think about that, even if you don't use SQL.

howfuckedismydatabase.com[0] is still as accurate as it ever was.

[0]: http://howfuckedismydatabase.com/nosql/

> Thank you NoSQL

Diversity and competition makes for better outcomes. I was never on the NoSQL bandwagon but even then I could see the benefits for its users. Namely schemaless documents for rapid development and relatively easy horizontal scaling. From those we got things like Firebase and a slew of NewSQL databases. Some SQL databases now have horizontal scaling in their roadmaps.

> It's hard to correct people's understanding when they learn things wrong the first time

Don't have a good solution to this. Trouble is that the 'first time' is learning and any amount of hand waving and thought experiments isn't enough to refute what they've learned combined with what they've yet to learn. No pain, no gain I suppose.

Reply on news.ycombinator.com