Hacker News Reader

> Can you confirm this (because I don't have the 32-bit version installed)? It still stinks, especially because it "silently failed after hitting the threshold", but I personally would feel better about 10gen if this little story is true (i.e. they warn you about it not only in downloads page, but every time you run mongod).

Just as a data point, I installed MongoDB on 32-bit Ubuntu 12.04 from the Ubuntu repo. At no point during install or service startup does it say anything.

Installing from source may be different.

Just installed it on ubuntu 32-bit (apt-get install mongodb). I now have a 32-bit mongodb process running. Minus all of the noise from apt about setting up xulrunner and the 101(!!!) other dependent packages, here's the entirety of the startup messages I see:

    mongodb start/running, process 13660

Any app that has a warning message will fail to warn you if it's output is rerouted to a log file. This sounds more like an issue with the package maintainers not understanding the limitation and thus not putting in a warning message post install.

This is true, in the Ubuntu packages this is printed to the default log at /var/log/mongodb/mongodb.log. It is also abundantly clear from the documentation. I struggle to understand how one could deploy a new datastore in production without reading the "getting started" level of documentation or looking in the log at some point.

The 2GB 32-bit limit of MongoDB seems like a complete non-issue to me.

  Wed Sep 19 17:29:21 [initandlisten] MongoDB starting : pid=3765 port=27017 dbpath=/var/lib/mongodb 32-bit host=deepthought
  Wed Sep 19 17:29:21 [initandlisten] 
  Wed Sep 19 17:29:21 [initandlisten] ** NOTE: when using MongoDB 32 bit, you are limited to about 2 gigabytes of data
  Wed Sep 19 17:29:21 [initandlisten] **       see http://blog.mongodb.org/post/137788967/32-bit-limitations
  Wed Sep 19 17:29:21 [initandlisten] **       with --journal, the limit is lower
  Wed Sep 19 17:29:21 [initandlisten] 
  Wed Sep 19 17:29:21 [initandlisten] db version v2.2.0, pdfile version 4.5
  Wed Sep 19 17:29:21 [initandlisten] git version: f5e83eae9cfbec7fb7a071321928f00d1b0c5207
  Wed Sep 19 17:29:21 [initandlisten] build info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
  Wed Sep 19 17:29:21 [initandlisten] options: { config: "/etc/  mongodb.conf", dbpath: "/var/lib/mongodb", journal: "true", logappend: "true", logpath: "/var/log/mongodb/mongodb.log" }
  Wed Sep 19 17:29:21 [initandlisten] journal dir=/var/lib/mongodb/journal
  Wed Sep 19 17:29:21 [initandlisten] recover : no journal files present, no recovery needed
  Wed Sep 19 17:29:21 [initandlisten] waiting for connections on port 27017
  Wed Sep 19 17:29:21 [websvr] admin web console waiting for connections on port 28017

The issue is not the limit; it's the silent failure. 32-bit Oracle doesn't do that, it tells you it can't extend the tablespace by raising an exception.

I'm not trying to defend Mongo, but the reason it doesn't tell you is because you didn't ask for it. If you care about the data (i.e. it's not a log or something that's not very important), you ought to always use "getLastError" to see if your data was actually stored or not (some drivers, like mongoose (for Node.js), just let you specify a simple flag ("safe:true") that does this automatically).

A shitty default, no doubt. But it can be changed easily. And "most" drivers offer that. And you usually connect to MongoDB using a driver.

http://www.mongodb.org/display/DOCS/getLastError+Command

Shitty defaults are 90% as bad as hardcoded unfixable settings.

Defaults usually don't get changed, convention over configuration, etc.

The default for the Java driver changed recently to the Safe mode.

So it does happen.

> If you care about the data (i.e. it's not a log or something that's not very important), you ought to always use "getLastError"

Wait they are promoting this as a database. It is right there in black on yellow "database". I don't know about you but I suspect most people expect databases to try their hardest to protect data. That means also having safe default. As in when I install it and put data in it, by default it should try hardest to make sure that data doesn't get corrupted. If it doesn't, it doesn't deserve to call itself a database.

> A shitty default, no doubt.

This is not a "oopsie" this is a deliberate lie and misinformation in order to produce fast benchmarks.

Normally people look at logs when things go wrong. Do you look at the log anytime something starts successfully and seems to be working? You must spend lots of time looking at logs.

This is a message that MUST be displayed on the console when you install the server for the first time. It's too important.

Also, you learn a tool before going into production. I never went into "production" with mongodb. All I did was experiment with a toy project. I never needed to look at the log.

Do you look at the log anytime something starts successfully and seems to be working?

When I first install something new that I'm going to rely on - yes.

How do you know that you're going to rely on it if it's new?

I install something new. I kick the tires. It takes a long time until I decide I'm going to rely on it. I'll look at the log at some point, but not necessarily the first time I install something. There are better things to do at that point.

Maybe you should learn something from your failure.

Maybe you should make constructive comments instead of making assumptions about what people learn or don't (or what they know or don't, or what has worked for them or hasn't).

There are some really disrespectful people in this community.

I didn't say you didn't learn anything. But you seem to be making excuses for why you shouldn't know some very basic things about MongoDB. It uses a memory-mapped file. How can that be larger than 2GB in 32-bit system?

It has async writes...this is pretty well documented by 10gen and is also something noted by a lot of tutorials, blog articles etc. You should have known something this basic about a database so important to your business.

None of that would bother me in the slightest if you were not still here defending such basic mistakes and blaming them on 10gen.

Come on, database (or any other system) that just silently fails to add new data?

2GB limit is clearly mentioned in the logs, that's fine, but anyone that sees this would expect that DB would start "screaming" loudly wherever it can (logs, response to the user on EVERY communication with the server, during any select,insert and others) that it reached this limit.

Async writes are async, so there cannot be a response to the user on failure unless you explicitly check(getLastError). This is very well documented behavior.

> There are some really disrespectful people in this community.

Your article begun and ended with sarcastic remarks about the product. Realistically, what kind of response did you expect? The issues described in your article are very real, and very worthy of repeated discussion, but the article itself eschews discussion in favor of pontification, sarcasm and flamebait.

You have been a "technology manager" at Bank of America in North Carolina since 1998. Do you try new technologies such as MongoDB professionally? What are your qualifications to make such a vague statement? More importantly, what is the lesson to be learned?

I made them in other replies, sorry for being so opaque.

The fact that a 32-bit memory-mapped file is limited to 2GB is basic comp-sci. It is also reported by MongoDB everytime it starts. Furthermore it is noted in the 10gen docs in several places.

Async writes...also extremely well documented.

So the failure here is two things: failure to properly research and understand a technology critical to their business. And then failure to take personal responsibility for the first failure, and instead to blame the vendor for not building a tradition DBMS despite the documentation about the stark differences.

I use MongoDB in a side-project if that is important to understand.

You do know he sold search and indexing startup to LinkedIn, including a custom storage backend?

So important to my business? What business are you talking about? You are just confused.

All this time I've been talking about a toy app that I wrote to kick MongoDB's tires. If you don't understand that, there is no point in having a conversation.

I never had or have any plans to use MongoDB for any business.

Sorry, you are right I conflated your article with another one referenced in the submitted post. I feel at least as stupid as I am, I can assure you.

It's also on the download page.

http://www.mongodb.org/downloads

Every time I start something I tail -f the relevant log to make sure it starts correctly. Think of it as isolating a possible failure point as a sanity check.

Then this whole post sounds like a non issue. You couldn't be bothered to even look at the log file for your app because it wasn't important enough, then you shouldn't be irate when it doesn't work as expected.

Confirmed, like others below. Installed on Ubuntu via APT.

~$ sudo service mongodb start

mongodb start/running, process 7680

Yes that is true for me, at least using the 10g binaries for Linux (not the Ubuntu package). You get this warning everytime you start the engine. You'd probably have to look at your logs to see it. It is also referenced in the installation documentation.

Look guys, MongoDB uses a memory-mapped file. How can it be larger than 2GB on a 32-bit system? There should not need to be "warnings everywhere".

I think expecting end-users to know or care about the implementation detail of a memory-mapped file is even sillier than all the other silly expectations floating around this thread.

You are right I would not expect end-users to know this. I would expect someone who writes articles about DBMS to know this though.

What do you expect end-users to know about oracle or any kind of database tuning?

Maybe stupid question, but isn't there mmap64 and mmap2 with which you can map larger files? You are limited to 2 gigs at a time, but that shouldn't be a complete show stopper.

How could it be larger? You could use mmap64.

Why are people running 32-bit OSes in 2012 again?

I run one on my phone daily? Why are you asking a question with an obvious answer of: not everything is 64 bits yet.

Why would I need that when I have <= 4GB RAM?

When comparing e.g. 32bit firefox and 64bit one, the first runs much smoother on my system. So if I didn't have 8GB I would install 32bit system. (not to mention that all binaries take less space)