Hacker News Reader

Comment by lossolo | Hacker News Reader

I've just migrated one project from mongo to postgresql and i advise you to do the same. It was my mistake to use mongo, after I've found memory leak in cursors first day I've used the db which I've reported and they fixed it. It was 2015.. If you have a lot of relations in your data don't use mongo, it's just hype. You will end up with collections without relations and then do joins in your code instead of having db do it for you.

View on HN

Even if you want NoSQL I'd use RethinkDB over Mongo any day. Way better query language, real-time support, and relational/regular SQL-like stuff.

https://rethinkdb.com/docs/comparison-tables/

And the use case of this post is exactly what RethinkDB does better: "One of our services periodically polls the database and reads the list of running containers with the query..."

> don't use mongo, it's just hype

I'm kind of curious as to where this hype is. I've almost never heard anybody say anything positive about mongodb. All I ever see is people saying it's terrible / hilarious for various reasons.

It's massively successful in the entry level web coding world due to a combination of good marketing and the belief that it gives you unlimited scale 'for free' and everything 'just works.'

Not saying those are impossible goals but so far no database has managed to deliver that. Building a large-scale endlessly-scalable database is still very hard and very detailed and easy to screw up.

Their website is quite hype prone:

>The Standard for Modern Applications

>MongoDB 3.0 features performance and scalability enhancements that place MongoDB at the forefront of the database market as the standard DBMS for modern applications.

>Also included in the release is our new and highly flexible storage architecture, which dramatically expands the set of mission-critical applications that you can run on MongoDB.

>These enhancements and more allow you to use MongoDB 3.0 to build applications never before possible at efficiency levels never before attainable.

The people who wrote that are not the same people who write the database.

The standard for the golden age of crap.

Hang out on the freenode room for mongo and you will see that most people, working on production software, make inquiries that reflect a complete lack of knowledge, common sense and leave you facepalming with no hope in humanity at a large.

The HN crowd tends to insult it, but outside of HN people hype it up. People keep talking about how much they love the MEAN stack. It's huge in the hackathon crowd, due to them sponsoring many of them, and its low learning curve.

I wish people would stop using acronyms and realize Express/Angular/Node is just as good with something other than Mongo.

I can confirm. On HN, meetups, Twitter, etc. no one talks about MEAN stack any more because of Mongo and the Angular 1/2 split (and React's popularity). In the last hackathon I went to, like most "corporate sponsored" ones, you got a special prize for using it.

thanks for spelling that out for me

great incentive to dive into React further and ditch Mongo

if you are just messing around you might be interested in vuejs & rethinkdb. found it a pretty intuitive stack, much easier than react for me

Are people using React starter kits? The major issue I'd see with React at hackathons is the large amount of configuration you typically need to do before you get started. I dislike starter kits due to the additional complexity overhead, but I can imagine for a throwaway project they'd be fine.

I think this is part of the reason who RoR, MEAN and other such frameworks are so popular. They tend to be easier to setup and many developers love it. My own theory is there are so many bad devs out there so that contributes to the hype because so may devs end up adopting it.

Take it with a grain of salt...

MEAN isn't as easy to set up and I would not compare it to Rails at all. Rails is a backend framework—MEAN is just a bunch of technologies used together to have an API and a SPA. I'd actually argue, from personal experience, that newbies have a lot of trouble with MEAN because while there's a CLI tool MEAN doesn't have the community support, conventions, etc. that Rails does. Plus, you have to learn too many things at once.

outside of HN people hype it up

Just today CNBC announced its list of "the 2016 CNBC Disruptor 50 companies"[1]

MongoDB is #19. It apparently is:

   Big data's thought leader

[1] http://www.cnbc.com/2016/06/07/2016-cnbcs-disruptor-50.html

It's accurate. MongoDB is killing it in the Big Data space.

Because of its excellent integration with Spark/Hadoop and the schemaless nature it's very useful in the analytics space.

I like the Postgres/Express/Ember/Node stack, if not just for the acronym.

It got pretty hyped a few years back when it was new, and 'NoSQL' stuff was just coming out.

It was the hottest thing a 2-4 years ago. Hype is probably the biggest association anyone who lived through that and didn't drink the coolaid has.

When it first came out it was hyped big time. Every other article was about how great Mongo was. About a year later the fallout started bubbling up. Digg went down for one or two weeks because of switching over to Mongo-- Reddit saw an influx of users and now is what it is.

I recall that Digg moved at least some part of their site to cassandra. Did they in fact do both?

Like with any online community, Hacker News can be kind of an echo chamber where groupthink reigns and alternative points of view aren't encouraged. MongoDB hype has died down here, but there are still some people that are fans.

There are some things MongoDB does fairly well:

* MongoDB is really easy to use

* Document databases can be great and flexible solutions for some kinds of projects

* Documentation is fairly good so learning the basics isn't too hard even if you know nothing about it

* scales fairly well at the initial stages

* arguably quicker to get a project off of the ground with than traditions RDBMs, which might be the most important consideration for any startup even if a complete rewrite would eventually need to take place

That being said, I've used MongoDB significantly before and it wouldn't be my first choice for most types of new project: PostgreSQL probably would be

Idk about hype, but for node.js virtually every tutorial uses it. I think the stereotype is that postgres locks you into a data model but your needs vary drastically. I suspect mongo is harder to retool the schema than it is widely claimed, while pg/msql is slightly easier than is often claimed.

I doubt they meet in the middle, but both have great use cases. scaffolding out a quick isomorphic js app is great for mongodb for example. pg is faster and more robust.

its just tech, depends what you are optimizing for

https://xkcd.com/937/

About the only thing I agree with is how great their docs are.

* Mongo is only easy to learn. Beyond simple demos, it gets harder and harder to use as projects evolve i.e. you have to do a lot of work yourself imo this is a common problem with nosql datastores that isn't exclusive to Mongo

* "Document databases can be great and flexible solutions for some kinds of projects": Postgresql has been able to work directly with JSON for some time now. There are also other document datastores that are more reliable than Mongo

* Scaling with Mongo is difficult, specifically the crazy setup. Even if you set it up properly, the results don't tend to match the marketing https://aphyr.com/posts/322-jepsen-mongodb-stale-reads

* "arguably quicker to get a project off of the ground with than traditions RDBMs" unless you're using Meteor, I'm also going to disagree here. Most frameworks target a relational database by default. Developing by convention tends to get you off the ground much faster than using something more specialized and niche

But they do make great mugs. Who here doesn't have at least a couple of MongoDB mugs. I don't use MongoDB and still have a bunch from random conferences over the last 3-4 years.

> scales fairly well at the initial stages

Not sure what that means, but scalability is the worst thing about mongo (though my experience with mongo is all from ~2.5 ish years ago). As soon as your working set of indices gets bigger than memory, performance falls off a cliff and your entire app grinds to a halt. In my experience, mysql and postgres have a more gradual decline in performance so you have some time with only mildly degraded performance to figure out a solution (plus they have more options for tuning which can buy you more time).

The hatred for MongoDB mostly comes from the PostgreSQL supporters camp. The rest of us are just using whatever tool makes sense.

MongoDB is the fastest database I've ever used. The easiest to get running, stable and scaled out. The best documentation by far and has excellent integration e.g. Spark, Hadoop.

If you are doing Big Data it's a great tool in the arsenal.

It's fast because it doesn't actually save your data when you ask it to. It saves it later, when it feels like it, maybe. Also, the hate I have for Mongo is from first-hand experience with the wretched thing. The scaling is insane. The next jump after your first server is like 9 servers (2 repsets w/ 3 servers each w/ 3 config nodes). I don't have that kind of cash laying around for servers, especially when instance-for-instance, Postgres can handle easily 10x the write load of mongo.

RethinkDb does all of this better, and with good engineering and API design behind it.

    If you have a lot of relations in your data don't
    use mongo

Why would you use Mongo if you have lots of relational data? Why would you not start with a relational database for that?

I know Mongo has issues but it's never going to beat an RDBMS on relational queries.

Because then you start moving your RDBMS logic in the application layer. And AFAIK, this is not the place for it.

It will never beat it in non relational queries also. To answer why - it was suggested by someone that said it's new modern db standard, that NOSQL is clear winner then showed me comparison on their site with SQL queries, all looked like it was substitution for RDBMS but it's not.

I think this is a common pitfall though: people start off thinking they don't really have relational data and then realize they actually do. Now they have a pile of code integrated with a DB that doesn't do relations well and can't be ported easily and then encounter cool bugs like this. No bueno.

We start our apps with mongo, and design them with a migration plan to postgres. We've found it's very easy to rapidly develop the application with mongo due to it's flexibility. Once we understand where or app is headed and what our relationships actually are, we pretty much pull the plug out of mongo and stick it in postgres. If you build a reasonably intelligent query wrapper it's fairly effortless. That being said, we're thinking of moving our early prototyping to Rethink now that it's made some strides.

Meanwhile, normalized data is what gives me so much flexibility when using Postgres at the start of an app. I just store my data as generically as possible and usually all I need to change under churn is the queries.

Denormalizing on day 1 (Mongo) has you making guesses about your data access patterns at the worst possible time instead of just thinking about the data itself.

> Denormalizing on day 1

This has nothing to do with database choice. This is just shitty development. It's a strawman at best.

Can you explain what you mean? As far as I know, normalizing is a nonsensical process for a document store. You can only normalize a relational schema.

> You can only normalize a relational schema.

Normalization is just a method of organization to minimize repetition of data. It has nothing to do with efficiency of operation. This is perfectly valid code:

    person = {
        _id: "person123",
        username: "lloyd-christmas"
    }

    comment = {
        _id: "comment123",
        person: "person123",
        text: "This is how I start",
    }

You don't have to do:

    person = {
        _id: "person123",
        username: "lloyd-christmas"
    }

    comment = {
        _id: "comment456",
        person: {
            _id: "person123",
            username: "lloyd-christmas"
        },
        text: "This is also valid"
    };

Sure, a join is faster than the first one where you'll have to hit the DB twice. The point is that you don't have to START with denormalizing everything. I start with normalized data and do more DB reads than I need. I figure out how the application uses my data as I go along, and denormalize the pieces I need only once I need them and am confident I won't bump into consistency issues (my username isn't updating every 5 seconds). Through this process I realize what the actual relationships are in my application and how my app functions request to request. This allows me to better structure my data. This is a quick update in mongo and usually a couple of lines of refactoring in application logic.

Obviously this is just an MCVE. My original point was that I find this to be a drastically more flexible process than starting off relational.

"It has nothing to do with efficiency of operation."

To nit-pick, I think normalisation improves the efficiency of updates, as you only have to modify the one place where the piece of data lies.

You can normalize data in Mongo and replicate relational database features in your application layer.

I'm just curious how that's an upside to using a relational database from the beginning when your plan is to migrate to a relational database anyways.

Switch out "Mongo" for "Postgres" in your bulk paragraph and you have the same scenario but with less work on your part and more features to help establish your data model.

One upside I can see if it you're more familiar with Mongo where using a relational database slows you down.

Maybe my other comment might help clarify: https://news.ycombinator.com/item?id=11858939

The bottom level of our application layer is a query builder which is almost a drag and drop replacement between mongo and postgres. By the time that layer is built out, we know what our database needs to look like. I find that adding/dropping fields and models in mongo to be drastically faster than moving models around in postgres. The above example would obviously end up in the same structure, whether we started with relational or not. It was nothing more than demonstrating an iterative process where you don't need to START denormalized just because "that's why you use nosql".

We try to be as incremental as possible when building our apps, and have found that using nosql allows for 20 small refactors that often end up being 2 larger refactors with a relational db. We've just found that it ends up being a faster production process, and we end up with a much more application-specific database instead of just "This is a Person, this is an Address, this is a Comment". Sure, we know beforehand that the application will contain all those components. We don't necessarily know how they'll be used in a request-by-request basis, and whether or not they will actually end up being one-to-one, one-to-many, or many-to-many.

This feels like you lack an architect who can see the bigger picture of your applications. I don't mean that insultingly, but with experience you tend not to need that second system syndrome. Do you find there's less rewriting over time as you become more experienced? Or is it just the kind of projects you work on?

That's not what "second system syndrome" means.

I've actually observed the opposite, that the more experienced you get, the more rewriting occurs and the less you tend to plan things out ahead of time, until you're Google/Facebook level and just accept that you will be rewriting constantly.

I'm not necessarily sure that you actually rewrite more, but that you're more willing to accept the current status and plan on some rewrites.

> This feels like you lack an architect who can see the bigger picture of your applications.

Quite the opposite. We feel that going in with the assumption that you know where the application is going to end up is hard-headed. However, acknowledging that the situation will definitely change doesn't absolve you of planning it out properly given the information currently available to you.

> Or is it just the kind of projects you work on?

We build mainly internal facing or b2b apps in the medtech space. Given that we need to integrate with larger players that don't really care much for small businesses, we can receive slow response times for data/api requests from any external sources we deal with.

e.g., Recently we built an application for a home-town pharmacy where we were forced to use two databases; one under our control and one which was controlled by a pharmacy management system. We needed to update certain models in their database while reading from other ones. They promised to build out a few stored procedures that we needed. They flat out lied on a couple, and then quoted us a 8 month turn around on the other ones. We'd be stuck with them regardless, the expectation of a curveball like that allows us to rapidly adapt.

Obviously, the pharmacy's business model doesn't change very rapidly. Iterating over the app through a few of their business cycles tends to give you enough knowledge of what you can build them, as well as what they really want. Regardless of how much time we spend planning with them, they'll always leave us with some form of an XY problem that we'll only understand after they use the application for an extended period.

We don't deal with web-scale, so the problems generally encountered in the mongo complaint arena tend to be irrelevant to us. Given our use case, I think the decision is pretty reasonable.

> Do you find there's less rewriting over time as you become more experienced?

We've factored that into our development strategy, which is why I mentioned the query wrapper. That query wrapper paired with a DAO level that's reasonably database agnostic makes our transition fairly simple and quick.

PostgreSQL and jsonb might make it a more seamless migration that some may think:

https://wiki.postgresql.org/wiki/What%27s_new_in_PostgreSQL_...

JSONB values are indexable and queryable, so there aren't very many downsides.

> If you have a lot of relations in your data don't use mongo, it's just hype. You will end up with collections without relations and then do joins in your code instead of having db do it for you.

So... you are not against MongoDB but against NoSQL in general? I've used MongoDB and I've never ended up with lots of joins in my code. But I guess it all depends on the use case and how you've structured your data. Document databases are not a silver bullet.

ArangoDB in contrast to MongoDB is a ACID document store also with graphs.

> If you have a lot of relations in your data don't use mongo

Perhaps you should have been using a relational database from the get go.

Sounds more like your issue, not mongo's.

Contrary to what the name implies, most relational databases don't handle querying relation-heavy data well. If you need to hit plenty of relations in your queries, instead consider something that is optimized for that, like a graph database (or multi-model including graph).

I still want to know how anybody is making money off of data that doesn't have a bunch of relationships in it.

Reply on news.ycombinator.com