Hacker News Reader

Comment by stevencorona | Hacker News Reader

stopped reading the articled referred to amazon s3 a database. instant credibility killer. no thanks.

(p.s the amazon product you're looking for is dynamo)

I think you're the one who is mistaken. Amazon has multiple database products. S3 is one of them. In fact, some of the folks who developed S3 also developed BigTable for Google.

Dynamo is a new database product they have that is also a key value store.

Dynamo is not a new database product. Sure, their SaaS offering is new, but Dynamo has been used internally at Amazon since atleast 2007, when they published a whitepaper on it. Dynamo was inspired by BigTable.

Sure, when you break it down S3 is just a big key-value store, but Dynamo is a much closer comparison to BigTable.

http://www.allthingsdistributed.com/files/amazon-dynamo-sosp...

Yes, I've read the Dynamo paper too and I know all about it's history at Amazon. I give talks on this subject all the time.

My point was that you're being short sighted by taking away the credibility of the author for calling S3 a database, which many people do, and especially did in 2010.

Hey, Jeremy, you used to use Cassandra at reddit, no? I seem to remember it caused you many problems, is that the case? Have you used it/would you recommend it nowadays?

Maybe I should start a StackOverflow thread, or make a mini-site where people can opine about various technology products. I find myself looking for opinions every time I'm thinking of using a new piece of software for my business.

S3 has been around for a much longer time than Dynamo has been. Dynamo is a very recent introduction by Amazon. In fact, Dynamo did not even exist when this article was published. (Dynamo came out early '12, the article is from '10).

And when you think about it, S3 is just a massive distributed key-value store with simple key querying and an HTTP api.

Amazon's Dyanmo Whitepaper is from 2007, they have used it internally for a long time- it was inspired by BigTable.. not S3.

Source: http://www.allthingsdistributed.com/files/amazon-dynamo-sosp...

Dynamo cites BigTable as solving similar problems but it's not inspired by BigTable at all, architecturally.

What prevents S3 from being used as a highly scalable NoSQL key-value store?

latency (and costs)

Neither of those things prevents something from being a database.

S3 is an excellent key/value store for large values. It's also publicly available, which is nice.

For example, all the thumbnail images on reddit are stored in S3. Essentially the client is given the key and then they can go look up the value themselves, and since it is publicly available http, it works right there in the browser.

Your example (reddit using S3) is not a database use, it is a content delivery network use. For that, it is good enough, but I'd choose a different key-value store for database, and mostly because of the latency (been there, tried that).

Alright, I can give you a better example. Netflix uses S3 to store movies that haven't been rendered yet. It's definitely a database. The server says "I need to render this movie to the iPhone format, what are the movie bits" and the "database" (S3) returns the entire movie.

Also, I would argue that static content delivery is just another form of database. It's just a massive key/value store, there the keys are the files and the values are the contents of the keys.

Let me ask you this: What is your definition of a database?

Interesting. Would you say a file system is a database?

Generally, yes.

I don't think I could come up with a coherent definition of a database which includes low-durability NoSQL key/value stores yet excludes file systems. This doesn't necessarily mean that file systems are a database, however; it could be that in order for the word "database" to be meaningful it must be defined in a way that excludes some things commonly thought of as databases, similar to why Pluto was deplaneted.

Yeah, right, the definition of the database is fuzzy. As every technology, S3 has its pros and cons, and the latency is a con this time. Take a latency-intensive application (1), and it will feel miserably slow with S3. Take a storage-intensive application with not-so-frequent access requirement (2), and it will be just OK with it.

(1) any page-load that requires 3-5 sequential DB access (based on the results that are returned in the previous steps).

(2) Netflix's "let's store somewhere until we really want to use it" qualifies.

I have read a few posts over the years of people using s3 for exactly that. I don't have specific links anymore, but a Google search turned up an example: http://petewarden.typepad.com/searchbrowser/2010/10/how-i-en...

S3 is also used as intermediary data storage when running Elastic MapReduce queries over data in your DynamoDB tables.

http://aws.typepad.com/aws/2012/01/aws-howto-using-amazon-el...

Reply on news.ycombinator.com