Building a Better Submission Form | Hacker News Reader

Building a Better Submission Form

open.blogs.nytimes.com

53 pointsmeghan18 comments

Besides the scalability of non-relational data stores, would the increased flexibility in object creation and modification be enough of a reason to make the switch? Would object-relational mapping be a comparable solution?

It depends :)

If your data doesn't lend itself to an RDB schema, go with a NoSql/key value store.

I think this is a great example of a form mapping more cleanly to a key value store than a relational DB. Submissions to forms like this are documents in the sense that they need to be recorded as-is. If the user moves or changes their name down the line its actually a feature and not a bug that these records won't be updated.

Nifty, but I could see this causing aggravation on the backend (i.e. using the data from one generation of your form to another). Still, very flexible.

Entirely typical for the industry, they waste untold developer hours dicking around with a single feature when the could have just installed an open source CMS and gotten the feature (and all kinds of other stuff) as part of the package. Grade: D

Yea, cause the New York Times could totally run off a simple Drupal install...

Grade: F

Actually, I'm quite confident the Times website and their cluster of associated web services could, in fact, run off of a Drupal install. Care to share your thoughts on why that wouldn't work? How much experience do you have with Drupal? With building newspaper websites?

Full disclosure: I'm a co-maintainer of 8 contributed modules for Drupal and have 3.5 years experience building websites and associated services (with a variety of tools, including Drupal) for the 3rd largest newspaper conglomerate in North America.

http://www.djangoproject.com/weblog/2006/mar/29/scripps/

apparently several newspapers run off a Django install, including the newspaper that employed the web developers who wrote it:

http://notesofgenius.com/django-web-framework

I have no idea why you're getting so many upvotes. I have yet to see anything about NYTimes.com that would preclude it running on Drupal. It seems to be working for whitehouse.gov, fastcompany.com, thenation.com, economist.com, and quite a few other high traffic sites.

EDIT: And I also have done Drupal development for several high-traffic magazine and newspaper sites that get millions of monthly visitors.

I spent some time at the NYT. I've also spent some time working with Drupal.

Scaling Drupal is somewhat proven, and it seems to work for those sites. But, according to compete.com, the traffic figures (uniques) for March 2010 are:

    nytimes.com: 14,008,884
    whitehouse.gov: 1,878,251
    fastcompany.com: 1,537,072
    thenation.com: 491,232
    economist.com: 400,613

That is, nytimes.com gets almost 7.5 times the traffic as whitehouse.gov, and almost 35 times the traffic of economist.com. Oh, also, don't forget about.com, which, at 40,511,000 uniques in March 2010, has almost 2.9 times the traffic of nytimes.com.

So, there's that. Then, the NYT has some ... unique requirements. Drupal just doesn't cut it for various the reasons, the clearest of which is the need to support all the non-tech-savvy journos and editors who actually create the gargantuan amount of content published by nytimes.com. There is a reason a custom CMS was written.

Color me skeptical. Please explain how the technology for scaling to several million visitors won't cut it for several tens of millions? And do you really think that the other sites have editors and journalists that are any more tech savvy than NYT's are? Drupal is completely flexible and configurable, on both the public side and the CMS side.

I'm not arguing that Drupal is the best choice for NYTimes.com, but it's very doable, and hardly the ridiculous option that the OP made it out to be.

Well, having been bitten by Drupal in this way repeatedly (before eventually stopping using it altogether), the thing to notice about those million-plus Drupal sites is that nobody's logged into them. The same cannot be said for the NYT.

Drupal doesn't know how to cache anything at all other than for anonymous users. Start logging users in and your problems really get going around the 200-concurrent-users mark, not the 200,000-user mark. At 80-plus queries per page (which for Drupal isn't very many) the database just dies on its arse even at relatively trivial loads when you have multiple concurrent users.

Drupal doesn't know how to cache anything at all other than for anonymous users.

True for vanilla Drupal, but hardly true for Drupal + modules. You have to keep in mind that Drupal is not some black magic CMS...it's just a PHP framework. No matter what NYT is built in, they likely are employing the same kinds of caching strategies that you can leverage in Drupal. Even if you have 5 queries per page, you're going to need a caching strategy to handle NYT levels of traffic, and Drupal can most likely integrate the same methods. Simple examples include authcache and cacherouter.

Excellent point! Indeed, modules are an excellent way to extend Drupal. The NYT could probably write any they needed, and they'd probably rewrite some of the (excellent-quality!) existing ones to fit their unique needs.

What would be the point, again, of working with Drupal? A better photo uploader?

I'm not sure if you're replying to someone else, but again, I'm not at all arguing that Drupal is the best choice, just that it's not a completely crazy idea to use a proven, open-source, well-supported CMS like Drupal for NYTimes.com. It could definitely do the job, and I was responding to jcapote's ridicule of the mere suggestion that it could.

I should also point out that the Times CMS has needs beyond just publishing a web site.

A short list of points to consider are a sprawling detailed and curated taxonomy, aggressive editorial publishing schedules, community integration, feed generation for partners, topics, venues, articles, various multimedia types (slideshow, interactive, audio, video, audio slideshows...)

I could (but won't) go on. No CMS on the market or from the open-source community meet the very unique set of needs (as well as the ones you can't predict).

I moved elsewhere in the company from the CMS group as this was playing out. Previously we'd used a highly customized version of FatWire's CMS which wasn't sufficient for current and future needs.

Hope that helps clarify things.

(Hi bkudria, sorry I missed you at GoogleIO)

I can't resist the temptation.

"Detailed and curated taxonomy" - supported in Drupal core "Aggressive editorial publishing schedules" - Workflow module "Community integration" - OpenID, Facebook Connect, Organic Groups... "feed generation" - Views module "topics, venues, articles, various multimedia types" - CCK for the data entry and storage, Views + custom templating for the display.

All fair points... but not sufficient as I said :)

Reply on news.ycombinator.com