Comment by twblalock | Hacker News Reader

Comment by twblalock | Hacker News Reader

I think the NoSQL trend is something similar to what is going on with programming language trends (in industry, not in academia). Many commonly used languages were strictly typed and static, such as C, C++, and Java. Then, dynamically typed languages became popular because they are easy to use, although they give up a lot of the safety and static analysis capability of the older languages. Let's call that the second phase, and it is exemplified by Ruby and Python. Now, newer languages with type inference are coming out that combine the safety of the older languages with the ease of use of the dynamic languages -- Scala, Go, etc. Let's call that the third phase.

In the database world, we are still in the second phase, which is exemplified by databases such as Mongo, which have started from scratch and thrown out many of the good ideas embodied in older databases. I suppose that the new JSON capabilities of Postgres are an attempt to move database design into the third phase, which combines the ease of use of NoSQL with the many superior qualities of traditional relational databases.

There's definitely a lot of throwing the baby out with the bath water. ACID is hard on a distributed system. Let me rephrase that. Performant ACID is hard on a distributed system.

ACID and schemas are not actually coupled. You can have strict schemas without ACID and vice versa.

Relational stores without ACID are very hard to reason about. Any associations that are created with the primary entry may or may not be there at lookup. This is solved via documents.

I agree with your prediction that e will have document schemas enforced in the DB.

> There's definitely a lot of throwing the baby out with the bath water. ACID is hard on a distributed system. Let me rephrase that. Performant ACID is hard on a distributed system.

> Relational stores without ACID are very hard to reason about. Any associations that are created with the primary entry may or may not be there at lookup. This is solved via documents.

This is only solved via. documents for applications which don't need relational consistency or transactions, which is a minor percentage of web-applications.

Most applications of Mongodb that I've seen do not fall into that category, further MongoDB is marketed as a generic, not as a very application specific kind of solution..

As soon as you find your application does need transactional consistency between entries, you'll be up the creek making your own paddle - since unlike with SQL databases where ACID is at least supported MongoDB just gives you:

> $isolated does not work with sharded clusters. [0]

[0] https://docs.mongodb.com/manual/core/write-operations-atomic...

Yeah, completely agree.

I should rephrase: "This is solved by modeling your data as documents whether or not that is its natural structure."

ACID and schemas are not actually coupled.

Well, somewhat. The C in ACID (consistency) is impossible without schema's, because they contain the data validation rules. Without schema's you'll have to do all data validation in the application layer. Which is probably what you want, given that you're going schemaless, but it's AID at the database layer, not ACID.

There's also additional considerations that weren't touched in the article, and they're not functional but architectural.

Sometimes you have payloads that can't fit in a traditional RDBMS, and SQL/Relational don't cluster/shard well.

NoSQL bring convenient clustering/sharding to your data. Anecdote: an old team chose Cassandra solely for its admin-tools (which are really cool), even if we didn't actually need a NoSQL; I lost that battle, but I least we all learned something.

My pet theory is that the rise of automated testing was/is a big factor in the rise of dynamic languages.

Test suites guard against most things type safety does, so if you have a test suite anyway, the type safety becomes a net burden.

I don't remember much automated testing being done in those early days of PHP, and I don't see how your theory would explain the current trend back towards typed languages.

I think this is true.

Perl was one of the first hugely successful dynamic languages, though I will politely defer to Shell and TCL folks for a rigorous discussion of that claim. (:

Perl has been really serious about testing since early on, and, to my understanding, remains one of the most test-centric major languages out there.

On the other hand you could use static typing to avoid writing a lot of tests. Maybe it's a bit of preference thing, I don't mind either approach personally.

I think automated testing is more about integration and regression than code correctness. If you have a web app that connects to a Java backend, for example, static typing won't prevent issues. You need to test the API calls and the UI, and make sure the right requests are made, and the expected outcomes occur. This was done manually by QA people in the past, but it's obviously more scalable to automate it.

The third phase already exists. It's SQL engines e.g. Hive, Presto, SparkSQL, SAP Vora etc. on top of NoSQL and something almost every big company is using today.

SparkSQL in particular can "run all 99 TPC-DS queries, which require many of the SQL:2003 features".

Yet without ACID. So many look to me like a familiar skin on a different animal.

Reply on news.ycombinator.com