Hacker News Reader

Comment by lloyd-christmas | Hacker News Reader

We start our apps with mongo, and design them with a migration plan to postgres. We've found it's very easy to rapidly develop the application with mongo due to it's flexibility. Once we understand where or app is headed and what our relationships actually are, we pretty much pull the plug out of mongo and stick it in postgres. If you build a reasonably intelligent query wrapper it's fairly effortless. That being said, we're thinking of moving our early prototyping to Rethink now that it's made some strides.

View on HN

Meanwhile, normalized data is what gives me so much flexibility when using Postgres at the start of an app. I just store my data as generically as possible and usually all I need to change under churn is the queries.

Denormalizing on day 1 (Mongo) has you making guesses about your data access patterns at the worst possible time instead of just thinking about the data itself.

> Denormalizing on day 1

This has nothing to do with database choice. This is just shitty development. It's a strawman at best.

Can you explain what you mean? As far as I know, normalizing is a nonsensical process for a document store. You can only normalize a relational schema.

> You can only normalize a relational schema.

Normalization is just a method of organization to minimize repetition of data. It has nothing to do with efficiency of operation. This is perfectly valid code:

    person = {
        _id: "person123",
        username: "lloyd-christmas"
    }

    comment = {
        _id: "comment123",
        person: "person123",
        text: "This is how I start",
    }

You don't have to do:

    person = {
        _id: "person123",
        username: "lloyd-christmas"
    }

    comment = {
        _id: "comment456",
        person: {
            _id: "person123",
            username: "lloyd-christmas"
        },
        text: "This is also valid"
    };

Sure, a join is faster than the first one where you'll have to hit the DB twice. The point is that you don't have to START with denormalizing everything. I start with normalized data and do more DB reads than I need. I figure out how the application uses my data as I go along, and denormalize the pieces I need only once I need them and am confident I won't bump into consistency issues (my username isn't updating every 5 seconds). Through this process I realize what the actual relationships are in my application and how my app functions request to request. This allows me to better structure my data. This is a quick update in mongo and usually a couple of lines of refactoring in application logic.

Obviously this is just an MCVE. My original point was that I find this to be a drastically more flexible process than starting off relational.

"It has nothing to do with efficiency of operation."

To nit-pick, I think normalisation improves the efficiency of updates, as you only have to modify the one place where the piece of data lies.

You can normalize data in Mongo and replicate relational database features in your application layer.

I'm just curious how that's an upside to using a relational database from the beginning when your plan is to migrate to a relational database anyways.

Switch out "Mongo" for "Postgres" in your bulk paragraph and you have the same scenario but with less work on your part and more features to help establish your data model.

One upside I can see if it you're more familiar with Mongo where using a relational database slows you down.

Maybe my other comment might help clarify: https://news.ycombinator.com/item?id=11858939

The bottom level of our application layer is a query builder which is almost a drag and drop replacement between mongo and postgres. By the time that layer is built out, we know what our database needs to look like. I find that adding/dropping fields and models in mongo to be drastically faster than moving models around in postgres. The above example would obviously end up in the same structure, whether we started with relational or not. It was nothing more than demonstrating an iterative process where you don't need to START denormalized just because "that's why you use nosql".

We try to be as incremental as possible when building our apps, and have found that using nosql allows for 20 small refactors that often end up being 2 larger refactors with a relational db. We've just found that it ends up being a faster production process, and we end up with a much more application-specific database instead of just "This is a Person, this is an Address, this is a Comment". Sure, we know beforehand that the application will contain all those components. We don't necessarily know how they'll be used in a request-by-request basis, and whether or not they will actually end up being one-to-one, one-to-many, or many-to-many.

This feels like you lack an architect who can see the bigger picture of your applications. I don't mean that insultingly, but with experience you tend not to need that second system syndrome. Do you find there's less rewriting over time as you become more experienced? Or is it just the kind of projects you work on?

That's not what "second system syndrome" means.

I've actually observed the opposite, that the more experienced you get, the more rewriting occurs and the less you tend to plan things out ahead of time, until you're Google/Facebook level and just accept that you will be rewriting constantly.

I'm not necessarily sure that you actually rewrite more, but that you're more willing to accept the current status and plan on some rewrites.

> This feels like you lack an architect who can see the bigger picture of your applications.

Quite the opposite. We feel that going in with the assumption that you know where the application is going to end up is hard-headed. However, acknowledging that the situation will definitely change doesn't absolve you of planning it out properly given the information currently available to you.

> Or is it just the kind of projects you work on?

We build mainly internal facing or b2b apps in the medtech space. Given that we need to integrate with larger players that don't really care much for small businesses, we can receive slow response times for data/api requests from any external sources we deal with.

e.g., Recently we built an application for a home-town pharmacy where we were forced to use two databases; one under our control and one which was controlled by a pharmacy management system. We needed to update certain models in their database while reading from other ones. They promised to build out a few stored procedures that we needed. They flat out lied on a couple, and then quoted us a 8 month turn around on the other ones. We'd be stuck with them regardless, the expectation of a curveball like that allows us to rapidly adapt.

Obviously, the pharmacy's business model doesn't change very rapidly. Iterating over the app through a few of their business cycles tends to give you enough knowledge of what you can build them, as well as what they really want. Regardless of how much time we spend planning with them, they'll always leave us with some form of an XY problem that we'll only understand after they use the application for an extended period.

We don't deal with web-scale, so the problems generally encountered in the mongo complaint arena tend to be irrelevant to us. Given our use case, I think the decision is pretty reasonable.

> Do you find there's less rewriting over time as you become more experienced?

We've factored that into our development strategy, which is why I mentioned the query wrapper. That query wrapper paired with a DAO level that's reasonably database agnostic makes our transition fairly simple and quick.

Reply on news.ycombinator.com