Data Model = Database
mikeomatic.net
mikeomatic.net
In this approach, no application can change data in the underlying tables directly. Change can only be done by calling appropriate stored procedures. This rule is not optional but mandatory and is enforced by database permissions given to the database account used by each application: grant access to stored procedures, deny DML access to the tables.
As an example, if customer name needs to be updated, client applications will call procedure called update_customer_phone (customer_id, new_phone) instead of issuing a direct SQL statement like
update customers set phone = new_phone where customer_id = XXX
There is no need for the application to know that table named customers even exists.
Read access should be provided through views and not to tables directly. Views provide abstraction layer that will insulate applications from underlying table changes.
While this might seem like a lot work, this approach ends up saving a lot of headache in the long run, especially when one database is used by multiple applications.
The main problem with this approach is unfortunately company politics. In many organizations, the only people who can change stored procedures are DBAs or a group of "database developers". The applications themselves are maintained by "application developers". Usually these are separate groups, reporting to different people with different priorities and getting them to work together is often times challenging.
I agree with this blog post, but this is clearly a problem that comes up all the time. My question is: what approaches to abstract data models has the industry already developed to deal with these problems of schema change and so on? I have a feeling it's pretty big business.
Also, web services repeat the problem of schema brittleness, because they have a schema... if that changes, it ripples through all the services that use it.
EDIT that is, unless you can automatically regenerate the entire service, based on the schema - this is cool and works for some cases (eg. forms), and it seems to be the direction the industry is headed (as a no-coding solution, with many benefits: no tedium, no errors, no expensive/unruly expert developers).
BTW: the blog author (Mike Arace) is also the developer of the (free) software he's promoting (CodaServer). I have no problem with that, it's an interesting article. Perhaps that's why he didn't mention alternatives. http://18thstreetsoftware.com/aboutus.php
Querying RDF triples within reasonable performance constraints is very hard for the general case. I've tried a lot in that area. You just can't get around the fact that not using upfront knowledge about access paths to define the storage structures means to accept a huge number of joins. However, it's certainly a possibility to encode that knowledge in indexes instead of the logical data model.
Column databases seem to be promising for RDF storage and query.
What about the old solutions to this old problem? That's what I was asking about.
At some clients I have been to, they worked on a data warehouse schema in a star schema format that acted as a master and in some places the abstract model just lived in people's heads. My previous job was like that. These are people that ended up using RDF/OWL to put this into an ontology to get a better grasp on the abstract model.
One of the biggest problems I encountered was testing and change control. With the rules engine you could suddenly change and manipulate things (sometimes on the fly) - that didn't mean that you should....
Edit: That or you have stupid people building the Database. Remember just because SQL is easy does not make the scema unimportant.
One reason could be a desire to keep your database (somewhat or totally) normalized, while also being able to make queries against more natural (for the programmer) abstractions. It's rarely the case that all of a given user's information is stored in a single table (and for good reason... read up on normalization (http://en.wikipedia.org/wiki/Database_normalization) if you're not sure why), but it can sure save a lot of programmer time if your data model acts like it's all in a single table.
The case of multiple applications or systems talking to a single database isn't really all that different from a single application talking to a database, unless you only have one database call in your entire application. As soon as you're talking to the database in more than one spot in your code, unless you're careful, you start running into the brittleness/dependency issues described by the author.
Even if you're the only one working on a given application, in three months you'll have little chance of remembering every dependency created by every database interaction in your application. Abstracting things with a data model won't fix all of the issues addressed by the author of the article, but it sure does make it easier to address them.
Edit: There are a lot of DB tools to help you. Adding views let's you keep normalized schema's while simplifying reading data. Triggers can automate logging etc. I often see people using high level data views that end up giving them less power to manipulate data than the DB has.
PS: Why assume the schema is going to suck? Often it does suck, but that should be telling you to change the schema. I mean I know people that can't stand ugly code but they are willing to turn a blind eye to terrible databases.
I'm doing a bit of work on trying to implement an RDF store on top of HBase - has anyone thought about this and would be interested in throwing ideas around?
Use tiers or abstraction to separate changing layers.
I hate to dismiss it as simply a plug for the author's product, but heck, it ain't like this is a new problem. Why buy yet another product to add into the stack? Programming languages and tools are made to solve this kind of stuff as part of their job.