Hacker News Reader

Comment by nadaviv | Hacker News Reader

I make heavy use of SQL in my daily work. A lot of our business logic lives in the database as views, triggers and various functions.

I actually find that taking this logic away from the app code and into the database is one of the best architectural decisions we took.

I think the primary issue some people have with doing that is that they don't know working with databases well enough, and consider them a dumb place to store data, and nothing more. This is what makes it difficult to maintain.

One thing that I do agree with is that the SQL code isn't portable - but its the same as your Ruby/PHP/NodeJs/whatever code not being portable. I don't see that as an issue.

View on HN

From a software engineering perspective, this is entirely backwards.

Databases offer a great way of looking at your data.

However they have very poor stories around version control, rollout, rollback, separating metadata from data, and creating multiple parallel environments for production, qa, and every dev engineer. Yes, I know that database vendors claim otherwise, but compared to what is standard for most programming environments these days, it is a major limitation.

But it doesn't stop there. If you are lucky enough to get a busy site, you will find that the database is the inevitable bottleneck that can't simply be scaled by putting machines in parallel. The more work you push down to that, the sooner you'll hit that bottleneck, and the harder it is to fix once you do. When you've experienced, as I have, the joys of periodic outages because the database choked at a million dynamic pages/day, you'll value moving logic out of the database, the ability to put caching in front of it, etc.

Now to all of this the database vendors have a trump card, what happens if multiple applications access the database? If your business logic is there, then you're fine, otherwise you're reimplementing that logic multiple times! The thing is that in the 15 years that I've seen people making that argument, I've not encountered a lot of organizations opening up a business critical transactional database for random applications. And in practice it is easier and saner to put an RPC layer on top of of your existing app, in which case you get the business logic that way.

I guess I should somewhere note that I spent several years of my life with my primary job being to write fancy SQL. I was also the go to person on the software development side for "get us to scale the database better". I do not criticize databases out of a lack of knowledge of how to use them. I criticize them based on a large amount of experience with how to make the most effective use of their capabilities.

There's nothing terribly complicated about recursive queries and common table expressions, and for the use cases in which they're commonly used (hierarchical data being one of those) they perform significantly better than all of the popular SQL alternatives in almost every case.

I was referring to putting business logic in your database, and not any particular query technique.

Sometimes it just saves you huge amounts of time and code to do something in the database though. For instance, if you need to join against a remote data source, you can:

1. Write a bunch of code to pull the data from both sources and join it in the application. You have to pick a join algorithm at the beginning, and if your data changes you may need to rewrite it later.

2. Add a remote table in the database and just do the join there. The database can use statistics to pick a join algorithm for you.

I'm sure there are cases where you'd pick #1. But in many cases, the database offers a feature that allows you to implement a solution and call it "done" in minutes or hours where an application-based solution would drag on for days or weeks and carry a larger maintenance burden.

Please go to https://news.ycombinator.com/item?id=5934136 and look at the final paragraph.

Yes, I'm well aware of what databases can do for me, and have made a good living getting them to jump through hoops to do their thing.

However part of knowing how to use a tool well is knowing its limitations. And from what I've learned, a web-based CRUD application is usually better off with business logic in the web layer, and not in the database.

Also I have absolutely no idea why you brought up joining against remote data sources. However I've had to do it against multiple databases over the years. And in my experience with Access, Sybase, Oracle, PostgreSQL and MySQL, I found that for all but the simplest cases, #1 was preferable. For each of those databases I can name a case where I started with #2 with clear issues, and wound up with #1 working well.

Why? Because if you understand query performance, you can make #1 work well. And databases tend to do a horrible job of understanding what's going on with remote data sources and picking good query plans for them. Which can cause #2 to suck in hard to fix ways. (Plus with PostgreSQL specifically there were security issues that made the DBA unwilling to open up one database remotely to the other. He was willing to let my application access both, but didn't want to allow other applications to be able to do it.)

That said, most people who need access to remote data are in a simple circumstance. And I know a lot more about query performance than your average software developer. So if #2 can work for you, you should go that route.

"will find that the database is the inevitable bottleneck that can't simply be scaled by putting machines in parallel"

In context, the alternatives available are something like:

1. Issue one recursive query 2. Issue many non-recursive queries 3. Use some kind of denormalized structure that is tuned to the one specific recursive query in question, but might perform badly on other queries.

#3 can work if you are focusing on the optimization of a single query and can pick a structure tuned to that one query. You can even parallelize it, because you can know the partitioning scheme that will work best for that one query. But outside of those constraints, or if the query profile changes over time, it does not work well. You end up needing to duplicate the data into many different structures to accommodate many different queries.

And I don't see how #2 has any advantage over #1 performance wise. Add caching layers or other performance hacks as needed.

For further context, I'm responding to someone who is advocating putting your business layer in your database.

I'm not against using tools that exist to make accessing your database more efficient - including recursive queries. I'm against making your database responsible for more of your application than it needs to be.

Why can't you version control database code/schema? I can do it fine using Rails migrations. I've also been looking at http://sqitch.org/ as well.

Why can't you create parallel instances for each dev/environment? (If licensing costs are a problem, maybe look at a free one?)

Why can't you version control database code/schema? I can do it fine using Rails migrations. I've also been looking at http://sqitch.org/ as well.

If you're willing to use a third party tool to manage the database, and ONLY use that tool, then you can solve version control/migration. I've used and written such tools myself.

However once you have software development and database administration separated, the DBAs generally need access to the database in a way that does not go through your tool / software development cycle / etc. And once you have random things in your database that your tool knows nothing about, you're on your way to maintenance headaches.

Furthermore third party tools like this generally want your site to be offline during database migrations. But if you've got a production site going 24x7, then you really want to figure out how to roll out upgrades without downtime. This is much, much easier if you can upgrade database and website independently of each other, with the database going first. That important detail tends to be poorly handled in my experience. (Though things change and I'd love to hear if that had improved.)

Furthermore you have the headache of identifying the difference between data tables and metadata tables. That is, some tables contain data that gets modified on the fly during normal use. (eg user information) Other tables contain data that is really application configuration. (eg allowed statuses for something)

Why can't you create parallel instances for each dev/environment? (If licensing costs are a problem, maybe look at a free one?)

You can. The difficulty lies in keeping them synchronized with what is in your repository. And also in providing ways to copy user information to the private instances so that devs can reproduce production problems. (Doubly so when your production problem is a performance issue that requires a copy of production data to recreate.)

These problems are all solvable. I've solved them. But the general story that databases provide on these issues does not match the simplicity and flexibility of having databases focus on providing a common data structure, and the application being stored elsewhere in your favorite source control system (eg git).

Why would this happen? Database changes should be treated just like any other code update.

Furthermore third party tools like this generally want your site to be offline during database migrations.

They do? Why?

I don't see why this is a problem at all (or worth mentioning).

The difficulty lies in keeping them synchronized with what is in your repository.

This is why you use migrations/tools/etc for managing database updates.

And also in providing ways to copy user information to the private instances so that devs can reproduce production problems. (Doubly so when your production problem is a performance issue that requires a copy of production data to recreate.)

I'm not sure how this is related.

But the general story that databases provide on these issues does not match the simplicity and flexibility of having databases focus on providing a common data structure, and the application being stored elsewhere in your favorite source control system (eg git).

I have no idea why you wouldn't want the database change scripts/functions/schema to be stored in git, alongside the application code.

> I have no idea why you wouldn't want the database change scripts/functions/schema to be stored in git, alongside the application code.

I couldn't agree more. It boggles my mind how many people are resistant to storing this stuff in version control. It is code! Treat it like code! The exact same thing applies to configuration management. Your chef cookbooks should live in your git repository right next to your application code, and right next to your database code. That's the only possible way for every node in your source history graph to be functional in isolation. If your database changes independently of your version control, you simply cannot check out an arbitrary commit and expect anything to work.

Use case for you based on a real example. Early Monday morning, your site goes down under load. You really want a DBA to be able to hop on it, identify that the cause is, for instance, a poorly executing query, and lock a decent query plan in place.

Oracle lets DBAs do that, but I've not seen a database migration tool that knows how to understand/express that.

Secondly when it comes time to update your schema, the easiest solution is to simply lock tables and do updates. However locking the table can involve a forced outage. If, however, you limit yourself to very specific kinds of updates, then you can update your schema without major outages. Now rolling forward is cheap. Rolling it back is very, very expensive. Tools generally just try to do what you asked, and don't know how to say, "Really?"

If you pay attention to this, then you want to be able to just roll forward. And you want to be able to roll your database forward independently of the web server. Now you have the option of rolling the database forward, then rolling out a new version of the web server code. Should you have an emergency problem, roll the web server code back, and leave the database on the new version. (This only works if the database is "dumb".)

One strategy to support this is to let the DBAs be in charge of the schema, separately from your development tree. Developers are allowed to create patches that alter the database on dev, and will submit those patches to the DBAs. If the DBAs do not object to the change, they are in charge of putting it on QA or production. (If the DBA does object, of course, it goes nowhere.)

And now the pattern is that DBAs maintain the official definition of the database. DBAs are willing to apply proven patches to production, and can make QA a clone of production whenever they want. In emergencies, DBAs can fix problems directly on production. (You try to have this happen as little as possible.) All developer changes to the database schema have to be blessed by DBAs. Database patches need to be applied in production separately from and strictly before release.

I've seen a number of different strategies for handling database schemas and code. This is the one that, in my opinion, worked best in practice.

Oracle lets DBAs do that, but I've not seen a database migration tool that knows how to understand/express that.

I'm assuming that Oracle lets you do this via some sort of a sql file/script? I'd think the process would be:

1. Create SQL file that locks the query plan

2. Test it

3. Check into version control

4. Using migration tool, apply to production database

PostgreSQL lets you do all sorts of updates without locks, fwiw, and that's what I'm most experienced with. And all postgresql administration can be done via text files that can be checked into version control.

That would be interesting - but I wonder if it makes sense.

@btilly: what is the name of the thing with locking queries you referred to?

Any upgrade script that does certain operations will lock tables as it goes. See your database documentation for details. This can include rebuilding indexes, removing columns, modifying existing columns, etc. Furthermore when you've got both stored procedures and tables changing, you have to get the order of operations correct. I'm not aware of any automated tool that can figure out the dependencies and do so correctly. (Mind you I have not looked recently.)

I'm assuming that Oracle lets you do this via some sort of a sql file/script?

Not that I know of. When I took the SQL tuning class where I learned about it, they had an interactive admin panel for tracking down, identifying, and fixing query plans.

But I'm not a DBA, I haven't used Oracle in a few years, and I do not know what all is available there.

In general, "ordinary" software has these advantages for the simple reason that it's ephemeral. The software starts with a clean universe each time you kill it and relaunch. That makes lots of things much, much easier.

Databases don't get to do that. The data is persistant between instances, between sessions and between copies. It never goes away and the database must always consider all other considerations secondary. If you want to change the object model of a Ruby program, you change the source and relaunch. If you want to change the schema of a database, you will need to provide the database instructions on what you want to do with all the existing data that is now inconsistent.

If you look at the hoops people jump through in always-on software, as opposed to "restart with #e33cf4" software, it's similarly gnarly.

Relational databases are kissing cousins with heavy-duty type systems. They make you do homework and eat your vegetables to forestall classes of error lots of people have never encountered and so discount to zero. Then it's decided that they aren't agile and fwoosh, out they go.

Absolutely.

Which is why I find it works well to have code flow from dev to production, and databases flow the other way. With all changes starting as well-tested patches that are then applied in production and propagated elsewhere.

In my experience: most people that don't know proper DB design and SQL will do two things:

- Over-utilize stored procedures, views, etc.

- Implement many obscure "hacks"

There is nothing wrong with using built-in database functionality, but it can be over done.

Reply on news.ycombinator.com