Ask HN: HN web stack? | Hacker News Reader

Ask HN: HN web stack?

I'm curious about the HN web stack. I know they use arc as the icing but I have no idea what is underneath.

server: Apache(mod_scheme) or plt(racket) web server are my two guesses db: MySQL, PostgreSQL - I read somewhere that they use MySQL

View on HN

It's running the Arc web server, which has its own implementation of the HTTP protocol. I don't think there's anything behind or in front of it. Probably not even Memcached. Think of the app serving HN as a framework. Here's how a "URL" entry would be implemented in Arc:

(defop hi req (prn "Hi there"))

http://news.ycombinator.com/whoami will return your login and IP. It's probably implemented in this fashion:

(defop whoami req (prn (req 'ip)) (prn (req 'id)))

Regards

Joe

No database. Just flat files cached in memory on demand, IIRC.

I think about that when people worry about scalability. HN seems to do okay.

using flat files just seems crazy to me, but HN seems to be doing it well.

Why? A database is just an optimized flat file with a query engine.

Because a database is an optimized flat file with a query engine. Its commonly the case that you wish to optimize your flat files' performance and query them. I believe that pg has mentioned a few times that he's 'notorious' for using flat files for this kind of thing. I'd love to see the code for Viaweb and the design decisions in there.

HN has scaled, but not because pg hasn't worried about scaling (although to be fair, it doesn't seem to have worried much):

"I already have problems enough with that. Remember, the original motivation for HN was to test a new programming language, and moreover one that's focused on experimenting with language design, not performance. Every time the site gets slow, I fortify myself by recalling McIlroy and Bentley's famous quote

The key to performance is elegance, not battalions of special cases.

and look for the bottleneck I can remove with least code. So far I've been able to keep up, in the sense that performance has remained consistently mediocre despite 14x growth. I don't know what I'll do next, but I'll probably think of something."

http://paulgraham.com/hackernews.html

Sure. Profiling and incremental tuning has been sufficient so far. It will probably be hard to scale indefinitely, but HN already handles quite a bit more traffic than many sites.

http://www.arclanguage.com/install

You can read the source code yourself. I don't remember the database, but it's running on an Arc server.

I have taken a look at the source, but wasn't sure if they were using all of it in production. If they are then quite impressive.

Being proxied to Server:Apache/2.2.8 (Ubuntu) fyi

edit: I don't think the page itself is actually proxied it seems only the static content is.

If the app is behind apache and writing static files for serving things up, and only hitting arc on operations like votes and cache misses then of course it's gonna scale! Especially if the app is read heavy :)

Is the HN source code actually open source itself?

yes its in the arc source

I thought the database was flat files.

Yes, everything is a file in UNIX

or OP could be trying to hack hackernews

And what a crisis that would be. All of our publicly-available comments would become available publicly!

How about all of our hashed passwords- that would be a treasure trove

http://openid.net/

I don't see why you're being downvoted, your comment is obviously sarcastic.

Reply on news.ycombinator.com