Hacker News Reader

Comment by tekmaven | Hacker News Reader

I’m not really sure they articulated why they had to switch very clearly. They didn’t like managing Mongo. They said they couldn’t use Mongo’s hosted solution BUT they switch to hosted Postgres. Why not just overcome the limitations preventing them from switching to hosted MongoDB?

View on HN

They probably mean that they wanted the DB hosted within the private subnet of their VPC. With a traditional hosted offering you’d connect over the public net to a system hosted by the SaaS provider. With RDS they could spin up the DB within their own VPC.

Couldn't they do this by using Atlas and choosing AWS as the host?

Atlas runs tne hosts inside mongodbs AWS account, we have the same restriant, becuase of client privacy isses and gdpr compliance we can let a 3rd party host the data, amazon is ok beacuse we control tbe data on tbe instances.

Client privacy issues should not preclude the use of MongoDB Atlas... Atlas offers encryption of data at rest as well as the ability to manage your own keys.

The article says the work began in July 2017. Was it available at that time?

It looks like they wanted to run Mongo inside their own Amazon account- under their direct control. AWS could do that with Postgres while MongoDB couldn't.

That makes me sad. I would love for AWS to have a hosted Mongo solution or some type of NoSQL database besides DynamoDB.

Do people use ElasticSearch as a primary data store? In my limited experience implementations don’t treat it as the source of truth.

Yes, but it only works if absolute-goddamn-integrity is not a concern.

We unfortunately inherited a cluster from someone who thought it would be appropriate as a SoT for forensic data, which causes me no end of grief.

Elastic maintains a page documenting the current status of their "resiliency" efforts on Elasticsearch:

https://www.elastic.co/guide/en/elasticsearch/resiliency/cur...

ElasticSearch doesn’t have adequate data integrity to treat it as a primary data store for most use cases.

You could do what they are doing here

What's wrong with running a EC2 instance with Mongo in it?

Nothing, but they outline their problems in the article

they talked about NTP. which to me means Mongo on EC2 maybe?

That's what I understood they were doing and they didn't want to do that anymore.

Yep that's what we were doing, and the management software (OpsManager) was also running on EC2 instances. We messed up the VPC configuration so that NTP didn't work on some of the instances - which unsurprisingly broke authentication between OpsManager and the db instances

The way I read it, mongo was hosted the same way postgres was - on AWS VMs. OpsManager wanted to be more like a SaaS.

I'm a little skeptical of the idea that you can do custom software development with a custom database schema and realistically expect to outsource DB management. But sure, you can try. And in any case, you'd hope a largely read-only and document oriented dataset like a newspaper's has a relatively simple schema; without too many crazy schema quirks.

Why wouldn’t you be able to outsource database server management? Are you referring to outsourcing schema management?

By server management - do you mean VM? Sure, you can outsource that. I mean that what and how a home-grown application uses a database tends to mean that database (software/schema/optimization/whatever) management cannot be application agnostic. So if you outsource this, you're either effectively hiring a consultant that still needs to deal with and learn details of specifically your application, or you should assume it's going to cost you some time to do yourself.

I'm sure you can get advice or buy know-how; but they're too coupled to think you're not also going to need to spend some time too. (At least: assuming your workload is large enough and complicated enough that naive brute force isn't an attractive option).

Everything you are saying is true, but when people talk about managed services, for the most part they are referring to someone else managing the VM, the operating system and the server application running on the VM - in this case the database.

Exactly! That's why I was confused - they didn't replace mongodb with colloquially "hosted" postgres; they stayed at the same level of (partial) hosting. I.e., they didn't "switch to hosted postgres", in any normal sense.

Reply on news.ycombinator.com