Comment by tcgv | Hacker News Reader

Comment by tcgv | Hacker News Reader

Parent Full thread

The article is very shallow on technical details indeed. It seems they are trying to promote MongoDB as the solution to their failed attempt of standardizing their application/data model:

> Historically, (...) HSBC did have an application core program environment, which had most of an application's core functionality. But it couldn't have a single programme environment running for all the countries, due to the differences in data models and databases.

> Another benefit is to use the same database for global data analytics and reporting. We don't need to translate into another data model or another database to run the analytics and reporting from that particular data.

So this "application core program" failed to provide an unified solution to their data warehousing needs, and now they believe they can solve this problem by unifying 65 databases into one MongoDB instance, "taking advantage" of its schema-less model.

It looks more like a global storage service than anything else to me.

So, problem: schema does not accommodate enough use cases.

Solution 1: refactor schema

Solution 2: abandon schema

The truth is that you can’t really abandon schema, it’s just spread through the code base now, and any violation of the schema will blow up not when you write malformed data but when you read it later.

Truth.

Anyone know any good tools to manage the evolution in schemas to address this problem?

At the lower level I’ve found ActiveMigration from ruby inspirational. Not using ruby but the idea is solid and I have built a similar system for C# and SQL Server.

You still need something higher level to oversee the schema, but I didn’t look closely - my schema is small enough I can keep track of it in my head.

Flyway is a good tool if you don't want to be tied to a specific language.

The community version (open-source/free) does have a few important limitations that may or may not matter to you.

+1 for flyway. Used it at two companies. Ability to fully reimplement schema from any version to the most recent version is amazing. Also the ability to review a schemas structure at a given git commit is super useful.

I am sure it is quite a complex decision. If I had to guess they have a bunch of document storage all over the place and they are unifying it.

As others have said, you can either validate the schema at WRITE time (RDBMS) or at READ time (NoSQL). Personally I hate the during READ time because then you have all sort of data integrity fun to work through... In either case, I am sure they have a ton of reference documents that they need to store, so easier to manage 1 Mongo than 64 different weird databases.

And if you use Owl you get both?

Reply on news.ycombinator.com