Oracle on why you shouldn't use NOSQL (PDF)
docs.google.com
docs.google.com
Hey Oracle, if you want to bring this point up, please at least pretend to compare the above costs between NoSQL and your own solution. Otherwise it is just pure FUD: "NoSQL databases eat your CPU cycles, disk and network badndwith! Booooo!"
as long as it has some backup in verifiable data
A discussion on TCO is very dependent on the context in which the analysis was made, such as application type, volume of data stored, volume of data indexed, type of data (e.g. is it a tree? is it a graph?), queries per second/hour, number of concurrent connections, analysis on further scaling possible (vertical / horizontal), analysis on the requirements for real-time ad-hoc queries, entry-level costs, and so on and so forth ...Basically, the variables are so many and so complex that any TCO analysis (regardless of validity) won't apply to your case.
That said - I rarely feel the need for NoSQL, simply because I can fine-tune PostgreSQL to whatever needs I want. But I'm not running neither Facebook, nor Twitter.
i.e. it is not that NoSQL systems are inefficient individually, it's that it is not efficient to maintain four separate resource pools.
(This is still possibly FUD nonsense: Oracle has to exist as an independent resource pool only because their virtualization licensing requires all potential CPUs to be licensed. Are any/all of the commercial NoSQL implementations licensed this way?)
You can get someone else (Google, Amazon etc.) to provide the hardware, administration and support of your database if you choose something else.
And you get more for less that you would with Oracle.
- NoSQL is good in scaling: but do you really need to scale? Is your business so great to handle 1M users? Simple MySQL (and PostgreSQL when mysql is not ok) will go a long way for many many installation.
- NoSQL are in good in scaling but it is pain to maintain it: it hard to hire somebody who knows it good, there are no good management tools, etc.
- NoSQL requires you to make your application logic more complex and not portable
- There is no clear NoSQL leader and clear mainstream knowledge. And a lot of these NoSQL implementation requires some secret sauce known only to engineers maintaining and developing big NoSQL installations at Facebook, Google and Amazon.
In short, for startups, NoSQL is one more variable introducing risks and unknowns. So I think startups should not go NoSQL unless that is the core of their offering.
Well said.
NoSQL may have a lot to offer in some cases, but I think "easy" is an illusion when you look at the full picture.
I actually agreed with much of the data integrity problems the paper pointed out, but found it a bit too condescending, or perhaps even a bit threatening, around the "You are not Google" part. Big Brother loves me!
I often run into people with the opinion that Oracle is the only viable solution.
http://www.oracle.com/openworld/connect/face-to-face/appreci...
(Previous Oracle Appreciation Concert performers have included Elton John, Billy Joel, Aerosmith, Roger Daltry, Don Henley, the Black Eyed Peas, Devo, and Joan Jett.)
I find this interesting. I wonder if there any real TCO comparisons have been done to this point.
Sure, this paper is full of FUD. But it does raise a valid question for me: What's the difference between paying for Oracle and paying for MySQL, Datastax, or Cloudera support/enterprise to get the added features needed to run my business and keep away from reinventing the wheel?
Then you would have to pay for the CPUs, the licenses and the consulting (lots of it, tuning a 100 TiB system is a huge task). Add another million or two...
How much would it cost with a NoSQL solution? Don't want to turn this answer into self promotion, but I'm confident it's well under the million with better performance (assuming you don't need a relational database).
What about the cost of manpower to operate one vs. the other? If you can hire one novice sysadmin @ $40k, labor costs will be small, but for a 150-machine cluster you aren't going to get far with a $40k novice.
In 2011, you don't need 150 nodes to build a 100 TiB system.
Don't take this as an official estimate, but I think we would advise on 7200 drives and more RAM and it would probably end up with a system one (several?) order(s) of magnitude faster than what you would get with a regular RDBMS.
Gives you up to 16Tb of ram in one system, but you'll pay. We just got two, they are insane beasts but cool.
Second, server motherboards are not limited to a mere 8 slots. 16's are widely available, even 32's can be found at retail, and there are more options out there than what Newegg or whatever you're shopping at has.
Unless you start with a million users generating data, NoSQL (or Oracle RDBMS) is premature optimization. You want to architect for growth just in case, fine, but don't waste your time before you know you will have a problem.
It becomes one, but only when your dataset reaches sizes almost nobody will ever see.
If your business model implies a scale-or-die condition, then I'd say it's fair to design your solution to accommodate NoSQL, but not waste time implementing it in production. It's a horrible waste of time to design something around a Cassandra cluster that will be hit once every second.
And, BTW, if you hit the performance architecture's limit (actually, you should have detailed monitoring to tell you how many weeks you have until you hit it), then you should consider what you'll do to solve your particular problem (which may not be what you anticipated initially). You should always invest a little time evaluating your options for different scenarios.
It is overkill to implement a full fledged SQL database and write a bunch of SQL queries, if all I need is a persistent key/value store.
I've been playing with Redis lately, and it is really incredibly easy to use, and it maps quite well to the data-structures that I already have in my code.
There is certainly use for both types of database, but a lot of the NoSQL products are genuinely good tools (completely ignoring scalability), and make using SQL look like hammering screws for certain applications.
I'm all for using the most appropriate tools, but sometimes path of least resistance is worth considering too. Android bundles SQLite as a basic service, while many/most organisations will have existing RDBMS SQL servers as part of their infrastructure which are well understood and maintained. In either of those scenarios SQL may well have more power than is required, but it's also already in place and well understood and on the principle of minimal deviation from the known standards, I'd probably still use it.
It allows you to do things can be arduous and annoying with a traditional SQL database.
What I wanted:
* Easy interface to a key/value store. * Reliability. Once something is stored in the database, I want a reasonably high level of guarantee that it won't be forgotten. * Trivial replication so if I server goes down and takes the database with it, I can restore it quickly and have a backup server in the mean time. * High speed on minimal hardware, so I wouldn't NEED to scale under likely usage scenarios. * Easy path to scaling if it should become necessary.
Every SQL database I know takes nontrivial effort to shard. CouchDB has built-in master-master replication that takes two seconds to set up.
SQL imposes performance penalties that I simply don't need, and so NoSQL is going to be faster on the same hardware, and require less scaling.
CouchDB makes things so simple and low-maintenance that I don't need to hire people who understand it, because I can keep it maintained in my free time. I haven't needed to do anything with it after I spent a couple days to deploy it, and it's happily replicating to a second CouchDB instance that I can fall back on if needed.
And if I get to the point where I need to scale, well, it will work fine to create more CouchDB instances (my data is read FAR more often than it's written, so simple replication should be workable).
Even with MySQL, I feel like I would NEED an expert to be sure to achieve all of my goals above -- not because I couldn't, but because it would waste too much of my time. With CouchDB I got something that Just Worked in exactly the way I needed it to.
I recently built a view in postgres that uses windowed aggregate functions (what Oracle would call "analytics") to present a certain summary of statistical data. The beauty of this is that it's declarative, and took me a LOT less time than coding up a manual aggregation over the specific windows and grouping that I needed, the unit testing surface area is a lot smaller, and it's written once and available to any client platform/language/framework/tools that can talk to Postgres.
Postgres and even moreso Oracle gives you this kind of power out of the box. If you're not using it, you're not getting the full value out of those products.
It sounded to me like he was saying that sometimes it's actually more work to figure out how to stuff your data into a relational db than a non-relational one?
This is tackled by quite a few startups, so it's not out of the realm of things we HNers do ourselves.
Mouse tracking can generate potentially millions of data points per day for even a small number of users. However, we only store a set amount of data per client.
If for example, you think that every point should be tracked individually you can easily run into the case where Postgres will be non-optimal under the read/write load.
Something like Redis on the other-hand might be optimal.
This is a case where its not a premature optimization, its just the right tool for the right job.
Then when the buffer reached 5000 requests, it would push the data in PosgreSQL using a COPY FROM STDIN. On committing the transaction, we also specified synchronous_commit == off
As far as DB tables go, each day a new table was created with a timestamp in its name, then a cron script would take care of tables older than one week, aggregating data and getting rid of junk.
This setup was handling tens of thousands of writes per second without a sweat on a pretty modest server. Of course, it's less than what Redis can do, but then again I trust PostgreSQL more than I trust Redis.
For example, a marketing manager wants to aggregate data set X against Y to see the outcome. The more data I have in structures that support these unforeseen and ad-hoc requirements the more insight my organization obtain. So personally for your situation I would front cache the data in an NoSQL type structure and bulk transfer it into a relational structure at set intervals to avoid having to write logic in an application layer for each use case that comes through the door. I know that there are emerging tools in this space for the NoSQL databases, but I still find analytical and reporting easier to do in the relational world, relational models seems to lend themselves better to discovering links between data sets after the fact.
I am curious if your hypothetical question is a reality. :)
You can see this page for an example of what I mean: http://www.clicktale.com/product/visitor_recordings
While I would probably agree that Cassandra was designed for scalability, there are hundreds of other NoSQL databases designed for completely different problems that have nothing to do with scaling. CouchDB, for instance, is really good at synchronizing data between separate disjoint databases (think temporary offline use); a problem some have with databases both big and small.
Every NoSQL database was born out of the need to solve a problem that was not well solved using existing technologies. Some of those problems are related to scaling, but scaling problems are just the cusp of what is now available to you to use. Definitely use SQL when it is the right tool for the job, but don't try to shoehorn it into the wrong job. There are other tools that just might be a better fit no matter how big your database is or how many people are going to be using it.
Sure. When your data is not relational (or, representing it as relations doesn't help you getting to it) you should consider other forms of representing it.
I am very fond of Zope and its underlying ZODB storage (and the connectors that allow it to store documents on mostly anything else), but it's very Python-centric and Python, as much as I love it, is not the perfect solution to every problem.
I suppose I should've clarified that sooner. Having had some experience scaling MySQL at growing businesses, I am now trying to anticipate an explosion of analytics requirements alongside an explosion of user growth and activity caused by feature growth.
I have built solutions that allow me to collect data from running applications, and later inject data from these collected scenarios into the business logic for testing and debugging.
All of this to say that any objective analysis of these systems would reveal no need for a relational database. In fact, imposing the relational transaction model onto these applications is clearly detrimental to the development of these systems, the cost of these systems, the testability of these systems (though this isn't really the fault of the database technology itself), and the performance of these systems.
We should be processing the information through an application designed solely according to the needs of the application, and then selective pushing data to a data warehouse for reporting (which can certainly be a relational database).
It is almost like Ford claiming all your transportation needs require a truck, and forcing all supermarkets to accommodate driving trucks up and down the grocery aisles...
Nobody would say you don't need trucks. But the fact that trucks are sometimes needed isn't proof you should always use a truck.
>>The NoSQL database has been sponsored by large internet companies. Such sponsorship has added credibility to internet scale claims of the databases and an aura of "coolness." The large internet companies have been motivated by having free database software that is under their complete control. These companies have hired top developers for their scalable database implementations and to keep them running. Is your company prepared to make such an investment in an area which is not your company's core competence? Do you really want to be contributing to an open source effort? Do you really want to bet your project on something that's on the bleeding edge? Do you want to build the superstar team required to make the NoSQL vision a reality? You are not Google.
In addition, I'd say that any reasonably-sized database requires full-time engineers to get decent performances (DBA in the case of Oracle).
Google employees surely got a good chuckle out of that paragraph.
Wow - a salient remark about the state of enterprise software and what message Oracle sees resonating with management.
DB operation is overhead. I can see wanting a superstar team to provide things that customers value, but no customer cares about SQL vs NoSQL.
I wasn't commenting on SQL vs NoSQL and which will add more value. I was making a remark on Oracle's insinuating tone "Your developers are probably poorly skilled, your not going to change that, using new tools require new skills, you'll fail at that because your not Google - don't even try"
Regardless of my personal opinions, what happened to striving for excellence?
That "insinuation" is mostly true.
> Regardless of my personal opinions, what happened to striving for excellence?
Striving for excellence in everything is bad business.
Excellence is expensive. In many cases, the expense far outweighs the benefits.
What it doesn't mention is that everyone who picks NoSQL generally acknowledges these downsides. So it doesn't do a good job explaining why people still pick NoSQL.
TCO is probably the biggest factor I would have in picking NoSQL. As a (Microsoft) SQL Server customer, we (developers) all hate the cost of scaling SQL Server for each customer. Each customer needs their own resource governance. This means either separate servers or a virtual machine, both tied to a SAN. Virtualization MSSQL licensing costs are large without much material benefit due to odd licensing strategies that negate most if not all financial benefit to virtualization for small IT shops like us. By comparison, the better we understand why customers use our product and the more engineering decisions we can make (get away with?) then the cheaper technologies we will be able to use (NoSQL, etc.).
The document makes a lot more sense when you imagine the Samuel L. Jackson character from Pulp Fiction reading it aloud.
As an alternative, how about having (Simpsons) "Comic Book Guy" read it? (with apologies to Steve Yegge)
Gee, thanks Oracle
That said, the NOSQL movement is so full of hype... It really annoys me that whenever anything new comes around, a large amount of people want to use it for EVERYTHING! NoSQL is inconvenient for a lot of what it is getting used for today... In 5 years, we'll be past that and we'll see it used more appropriately.
No way. What about System R? And there were probably others of which I am unaware. Many of the now-standard RDBMS implementation techniques were pioneered in System R. This paper has a good overview: http://www.cs.berkeley.edu/~brewer/cs262/SystemR.pdf
I'm actually very interested in what made the relational model so commercially successful, if you happen to know more about it (it's hard to judge now, because so many other features have been added and market standardization etc). The mathematicians say it was because it's based on a mathematical model - but that doesn't sound like it would be very convincing to pragmatic benefit-oriented business buyers.
EDIT that pdf looks pretty good - I'll have a proper read of it when I get the chance - thanks
They totally fail at argumenting them though, having a strong bias towards their products. As expected.
http://gigaom.com/cloud/big-data-startups-weigh-in-on-compet...
The point of this paper is to scare Oracle's customers from switching to NoSQL, and I agree that companies should not blindly switch from SQL to NoSQL without the willingness to relearn and adapt.
Really?! I never would have guessed. I mean, it's not like it's right there in the name.
They go on to state that if you need RDBMS features, you should use an RDBMS. My mind is being completely blown here.
"embrace asynchrony" - really?
"Clearly the traditional 3-tier application development model scales well and the desired development model for established non-internet companies." - there's a missing word here somewhere to make this a sentence
hipster oracle loves irony
With all my respect to Oracle (and gratitude for that part of living i make from it), they would better spend these resources on implementing a distributed query execution (the parallel query implementation they have is a joke really), and it would make their position vs. NoSQL much more stronger.
I just need PostgreSQL and SSDs.. done!