Are Commercial Databases Worth It?
codingthewheel.com
codingthewheel.com
I've always called this plausible deniability in corporate culture. "It's not my fault the DB is down. I'm on the phone with Oracle right now, they'll have a fix soon". The OSS solution is much riskier. "I chose this free DB, and now it's broken." In large corporations, the first is much safer for the employee, but much more expensive for the corporation.
As a half-serious idea for a company, I thought of IT insurance. Rather than paying support to Oracle, we will audit your solution and charge premiums based on the riskiness of your setup. In case your OSS DB goes down, the insurance company would pay the downtime.
The company would provide two services. 1) I expect our "plausible deniability" would be cheaper than paying Oracle, and 2) we put a pricetag on the reliability of your setup.
If you're not working on it you should :-)
I can't wait for the day when such systems become plug & play
Supporting your own database effectively means writing your own database. Once you start hacking at it to fix a problem it's a fork and you're on your own. Any organization that puts its eggs in one person like this is bonkers.
I'm not claiming that MySQL is perfect (or even good), only that many organizations think it's good enough, and are willing to take that risk.
I can think of only one app that was large enough and complex enough to really require a "commercial" db, and it was talking with SAP.... ohh, and it ran into constant problems.
What I don't understand is that other than features, the other argument is "Support". I have never seen microsoft get on the phone with a guy for free (yes you can pay...) and help out. But you know what, you can pay for a guy to help you out with postgres.
And you know, by going with proprietary databases, that cost so much money, there were severe restrictions on getting a database. They were in the hands of another team, who took weeks or months to get us setup. I know a part of that is server management, but another part was cost cutting on licenses. Remember that a small cause can have a very large effect in a corporate culture.
It's nice to know they are there when you're doing a hairy Exchange migration...
Can you get your money's worth out of a database like Oracle? I think you absolutely can if you USE the features it comes with. If you don't want to do that, or don't need those features, then you are throwing your money away.
There are projects where I really love using Oracle (and to a lesser degree SQL Server). It has everything to do with features that solve problems.
I loathe DB2 though. Also, there are some weird smaller commercial databases like Pervasive that seem to add nothing but difficult interfaces.
FWIW, the latest beta release of PostgreSQL supports recursive queries (using the SQL standard WITH RECURSIVE rather than CONNECT BY, but still).
The list of features that justify using a commercial database keeps getting shorter each year.
Have you ever been on a day-long 7-person email conversation to change one string to another in a text file, and 6 of those people were IT people? I feel your pain.
There's probably not much of an advantage in the database engine itself, but the SQL Server suite shined in 2 areas: analytics and reporting. The 2 add-on components made large-scale data mining & model-building much simple, and also automated the generation & delivery of complex reports tracking many metrics. I could probably do both with MySQL, but only with much more effort.
I know you could. You know you could. Most of the people here know you could.
But your boss doesn't want to bet on you. That would require him to manage you. Easier to spend a bunch of money through a capital expenditure request to get a piece of commercial software that already does it. When your boss(es) don't get the job done, it's easier to blame a vendor than an employee.
- Security
Securing data in a database is pretty important if you're dealing with any kind of sensitive data. Levels of sensitivity may depend on what you do and may vary over SSNs to sales numbers for next quarter. AFAIK (correct me if I'm wrong) - PGSql/MySQL don't really offer concrete solutions to protect data - apart from common sense like patches, secure passwords, et al. Now look at offerings from the big co's. Companies of any kind will feel more comfortable with these.
- Auditing/Compliance
Are you a public corporation? Semi-large? Do you have payroll or other info in a database? Enter Sarbanes Oxley! Easier to have solutions built by the db vendors rather than deploy 3rd party stuff.
You're comparing commercial databases favorably to Open Source databases on the grounds of security? This differs vastly from my experience. Oracle is an impenetrable mess:
- do or don't use TNSLSNR
- if you use TNSLSNR, do or do not secure it with password protection
- my experience with Oracle has been that port allocation is rather nondeterministic. I've attempted to perform identical installations several times and ended up with TNS and the databases on weird ports for reasons I can't explain.
- user management of oracle behaves differently on different platforms, i think. IIRC it integrates with NT credentials on Windows, and UNIX credentials for certain connection situations, but there's also a username/password file that controls who can connect
- you issue alter password commands in unquoted SQL. Because of this you don't get to use things like mixed case (?) or spaces or most punctuation characters
- but you CAN use quotes around the password, in which case it behaves sanely. I didn't even know this until I looked it up it's so uncommon.
- Oracle attempted to market Oracle 9 as "Unbreakable": http://news.cnet.com/8301-10784_3-5808928-7.html . They had enjoyed a pass from hackers since systems/network folks tend to bury the DB at the back of the architecture, rarely accessible on the public internet. After those shenanigans security minded DBAs had not the best time of their careers as they had dozens of security patches per quarter for a while.
Perhaps the commercial guys have security "features", such as row based security or encryption, that the free products don't have. While it's surely better to have those than not, I'm not convinced applying those proprietary features to an otherwise insecure design is satisfactory "defense in depth". You can, however, have sufficient defense in depth without those features.
Sarbanes Oxley isn't too bad: the crux of it is that you have to enumerate your controls and stick to them. I understand PCI compliance is another matter. I don't know much about it, though.
If a project truly requires ACID to the point there's a sensible worry about the disk holding the right data, I'm pretty sure I'd save money (and hair) paying for a medium server and a license to deal with that (particular) data. It's really rare to run into a problem that requires both that level of reliability and needs performance beyond what a medium server can reasonably handle. I think most projects are far far more likely to screw the data in some other way, so one could question if it's worthwhile.
I'm saying this having had to implement true ACID transactions (and replication) myself. It's a ton of work getting it right through all of the layers of libraries, os, filesystems, controllers, and drives. It's hard to trust user-level programs to be able to do it given an arbitrary stack below them. I place more trust in a system that can cut through some of those layers (or call across campus and control how some of those layers work) and work with particular hardware.
I place even more trust in someone screwing the data up at a higher level and it really not mattering nearly as much as people make it out to anyway. Even in "critical" cases like storing financial transactions, it usually just results in people having to call around or get in and fixing a few things.. which is WAY cheaper than the huge cost to lower the odds by 90%, and you've got to be prepared to do that anyway.
SQL Server is cheap, and like he said, it's miles better than the free options.
The real danger of using expensive proprietary databases is that because of the cost per box you tend to design your projects to use one big expensive server instead of multiple small, cheap servers. The big server approach eventually hits a scalability ceiling and then you have to pay for X more big expensive servers and re-architect a bit. This may sound absurd but I've seen it happen more than once. The killer feature of the free, open-source databases is as much the "free" part as it is the "open source" part.
2) Dev licenses are usually free - the development edition of MSSQL is a free download from MSDN
3) Far more absurd is "sharding", which IBM invented in the 80s and all the major vendors abandoned in the 90s.
We run a box that cost ~$4k all in ($2k hardware + the aforementioned software costs), and stash it in a cage that costs $400/month to keep it connected power and a fat pipe. Assuming it lasts us 4 years, that's $23,200 in server costs over the life of the box.
Now assume we'd skimped on a LAMP stack and pocked ourself a cool two grand in software costs. At the end of that 4 years, we'd still have spent $21,200 keeping our servers alive.
And it scales out exactly the same way. We can throw hardware at the problem every bit as easily as you. It's just, what? 9% more expensive over time, depending on your math. And that's on top of a ridiculously low price tag anyway.
Web servers are dirt cheap, regardless of which stack you go with. The price difference between commercial and free databases are just noise. In the end, it's certainly not something you should consider when choosing a technology.
The other point, though, is that it's not the $2k or $20k in software costs. Really, to a big enterprise, $20k is nothing. The real problem is that it ties you down to making architectural decisions because of the cost of the software. Say you're right and you can license MS SQL for only $2k. What happens when you start partitioning your data and end up with 50 servers? Suddenly that $2k is $100k... or if you're on the enterprise plan it's $20k * 50 = $1,000,000. And I'm not speculating. I've lived thorough this scenario more than once.
Also, I object to the phrase "skimped on a LAMP stack." It's not skimping. Cost factors aside, the LAMP stack is better.
- you have seen organizations pay $20k for SQL Server
- 50 boxes like that will cost a lot of money.
- SQL Server is actually worse than (at least one of) the free options
- I've actually purchased SQL server, bundled with the OS and all other necessary software for $2k.
- 50 boxes like that won't cost significantly more than they would with free software, if you consider the dominant cost of operating it, which is hosting fees.
- SQL Server is actually really good.
So you have someone to blame, who is tough to identify with open source software.
Never forget, in many large enterprises, these decisions are not technical, they are political.
Wrong, they're financial. The only factor is what the auditors will sign off on.
Not everyone is a startup. Not everyone cares about clever hacks and rag-tag scalability and bragging about how they scaled up to 10k transactions per second on YouTube. Sometimes you just want to throw $10M at the fucking thing and have it work so everyone in your company gets the data they need.
I simply don't care what rdbms I coding to, I can tweak to leverage quirkiness in T-SQL or PL/SQL if needed or advantageous to a project. And, <whispering>, PL/pgSQL.
A common experience for me when job searching is to apply for a job where, for example, they are primarily looking for Oracle PL/SQL development. I've been doing SQL coding for roughly 8 years, the last 2 years or so have been primarily in Oracle PL/SQL. I have been turned down for interviews with feedback that says "oh, we need someone with 5 years of Oracle development work."
If it's techies doing the resume reviews, I always get at least an interview. Sometimes even then, I have to sell the fact that SQL skills transfer nicely between rdbms harder than I should.
I'm not saying that the open source solutions can't be competitive from a technological standpoint, simply that they have to win on cost, availability of experienced people for dev and admin, and overcome what amounts to a late start in the market.
Plus, on the low end of the market, aren't there express editions of SQL Server and Oracle available for free? For lightweight use these would seem to be reasonable competitors to open source.
Unfortunately that's how the corporate world works.