Designing Web Applications for Scalability | Hacker News Reader

Designing Web Applications for Scalability

osconvo.com

10 pointsherewego3 comments

Preamble: I'm not being negative! I always appreciate any effort to elucidate complicating matters--especially those on matters I happen to have a vested interest in--but seeing as this is A Really Challenging Engineering Problem That Lot's Of People Like Myself Spend The Better Part Of Our Careers Trying To Solve and it's being offered up in a pithy, bite-size, 1000-word blog post on "Designing Web Applications for Scalability", well, I'm going to have to get out my red pen.

The entire subsection on "Choosing your data storage setup" is completely inconsistent precisely because it attempts to tackle such a huge problem in the space of a few paragraphs. Couch and Redis don't even partition out of the box, Tokyo Cabinet only does so with the help of Lightcloud, Mongo's auto-sharding is still unready for production, and Cassandra and Voldemort are fully distributed clusters for whom replication and partitioning mean very different things (and, incidentally, using "partitioning" and "sharding" interchangeably is one thing; doing the same for "shard" and "node" is very different).

And of course by muddying the waters with respect to the datastores themselves the entire treatment of designing a schema is similarly inconsistent. How can you have a universal keying strategy when you don't even know the terms of your discussion? Are you hashing to shards or to a ring? On the client side? With Zookeeper? Or with a messaging service?

Anyway, I wholeheartedly agree with the author's spirit of building for customer acquisition while also building for scalability, but I guess my advice would be "do plenty of homework and never expect to learn everything you'll need to know after reading a hundred blog posts--much less after reading just one."

Fair enough. I got about 1500 words into it and said to myself, this is a book I'm trying to fit into a post. Such is the reality of complex topics like these.

I agree with most of the points you're making, but I also think that its important to remember that most people have no idea about what's even involved in the basics. Your points are advanced and are details that are VERY important, but are beyond the scope of a post. If the criteria for trying to put into words some of the concepts behind scalability were that you need to be writing a dissertation to do so, then we would live in a sadly uninformed world.

My intent wasn't to make it a thorough primer so much as to touch on some of the concepts involved in making scalability a reality in day-to-day development of web applications: scalable datastorage, scalable application layers, etc. I hope the readers see it for what it is and then go do further research of their own.

Nothing stunning, no new thinking, but a good set of broad fundamentals.

Reply on news.ycombinator.com