SciDB – relational daddy answers Google, Hadoop, NoSQL
theregister.co.uk
theregister.co.uk
Very few applications I've ever designed ever seemed like a good fit when mapped to a relational data model - they always looked ugly to me and the resulting contortions required to map to and from in-memory objects pretty much always made me wish there was a better way (ORMs seemed to help initially but always became problematic in the long term).
The most common reaction from colleagues when I mention document oriented databases is "they will become a mess over time". Sorry, every large commercial application that I have seen that uses SQL databases is already a mess. Look at your average production ERP data model - they are usually things of Lovecraftian nightmares.
If he is amazed that Facebook can run with MySQL "40,000" ways, then I'm equally amazed that SAP can run at all! However, both seem to work fairly happily - just because you don't personally like a technology doesn't mean that others can't be perfectly happy with it.
Do you think that NoSQL will become more of a mess than traditional databases over a long period of time, so much so that it becomes unusable, or do you think the app relying on good data will keep the data clean?
Interestingly, I started my career on NoSQL databases in the 80's. We just didn't call them that at the time.
I think for many applications it's sign of bad design if the data model really fits the application in the sense that application data structures have 1:1 mappings in the database. It's similar to the relationship between the UI and the underlying object model (or whatever the paradigm may be). You need separation of concerns.
Data lives longer and usually serves more than one application over time. Fitting it too closely to the needs of one particular application at one particular point in time causes too many dependencies.
Of course, there are exceptions. Sometimes applications are trivial. Sometimes data isn't analyzed or used in other contexts. Sometimes performance dominates everything else. But for most applications, the infamaous impedance mismatch is an automatic consequence of acknowledging the different requirements we have for applications and data respectively.
The data should be architected for its own consistency, usability (and re-usability), and robustness. Of course for very complex needs the data will intrinsically become incredibly complex, which is why you should be using decent design tools to organize and document at a higher level.
Applications are layered on top (often many applications), and often there will be an impedance mismatch.
However what I hear a lot from the NoSQL camp is essentially that by hiding the complexity away into unbounded documents, the problem doesn't exist. That is, pardon the bluntness, unbelievably naive. Try to model one of those ERP systems and seriously -- have fun.
There's no big surprise that most of the success stories of NoSQL have been examples of intense simplicity scaled to large heights.
Not every problem is so simple. Few problems are so simple.
To your orginal comment -- while SQL is not the most elegant language, I like it enough. I love relational database design, however, as there is a purity and elegance in it that is very rewarding. In many ways it's like programming in assembly, and while it might not be for everyone, for some of us it's deeply enjoyable.
The ad blockers killed them...
VoltDB is focused on mutating an in-memory dataset at ultra-high speed with full transactions and ACID.
SciDB is focused on analyzing petabytes of scientific data in array form, where people care about things like data provenance and accumulated error bounds.
Almost polar opposites in use, but similar in philosophy. Both believe specialized systems are the way to handle specialized workloads.