Hacker News Reader

Comment by bogomipz | Hacker News Reader

I would like to hear more about the introduction of LSM-based RocksDB storage into the Parse architecture. Specifically how long before they felt MMAP was no longer meeting their needs?

I was also curious about this? >"Frequent (daily) master reelections on AWS EC2. Rollback files were discarded and let to data loss."

Was the velocity of rollbacks sufficiently high enough that they were't able to process them?

I thought Parse was a great company with a great product that served an important market. I'm still not sure why FB acquired them only to retire it.

View on HN

We had issues with the MMAPv1 storage engine from the beginning, mostly due to it's lack of document level locking and storage bloat.

One of the biggest benefits from being acquired by Facebook was working with the RocksDB team to come up with MongoDB +RocksDB, but this was also happening around the time when Parse was starting to wind down.

Being the only company running this new storage engine freaked us the fuck out. So we tried to blog and give talks as much as possible to get other folks interested and willing to test it out.

Parse now running MongoDB on RocksDB: http://blog.parse.com/announcements/mongodb-rocksdb-parse/

Strata: Open Source Library for Efficient MongoDB Backup: http://blog.parse.com/learn/engineering/strata-open-source-l...

MongoDB + RocksDB: Writing so Fast it Makes Your Head Spin: http://blog.parse.com/learn/engineering/mongodb-rocksdb-writ...

MongoDB + RocksDB: Benchmark Setup & Compression: http://blog.parse.com/learn/engineering/mongodb-rocksdb-benc...

I thought you might say that. If you have an app that does a lot of deletes that bloat becomes noticeable quickly. I think they might actually have document level finally. Rocks is interesting as it deals with the write amplification problem and its tuneable. TokuMX was another interesting storage and had good compression. I would curious if you ever evaluated that.

Are you with Baqend now? Is this your medium post?

Before even looking into RocksDB we tried TokuMX and even got pretty well aquatinted with their dev team. We ran into the same issues when testing Wired Tiger. Neither could handle millions of mongo collections. Since we were involved in the RockDB storage engine from the beginning, we made sure the implementation could handle that many collections.

This isn't my post and I don't work at Baqend, but I will say that the author comes off as a presumptuous asshat.

Yeah this reads like this person at Baqend is going over "lessons learned" while at Parse. That's why I thought it might be someone from Parse. It seems like bad form to write a blog post on what other people should have done differently and use that to publicize your own company. Parse was a success.

> So here are some facts and trivia that are not so well-known or published that I collected by talking to Parse engineers that now work at Facebook. As I am unsure about whether they were allowed to share this information, I will not mention them by name.

It's both lessons learned by Parse engineers and us, so I think the intended ambiguity is okay.

>"It's both lessons learned by Parse engineers and us"

All of those bullet points are from Parse's history. Where is Baquend's contribution to the "lessons learned" in this blog post then?

Reply on news.ycombinator.com