According to the link in dmgrow's comment, it's

    Score = (P-1) / (T+2)^G

where P is the number of votes, T is the time in hours since submission, and G is a power that controls the steepness/rate of decline in ranking, so popularity is decreases exponentially over time.

I'm sure that YC-affiliated posts have a different ranking algorithm, considering you can't vote on them.

Some other cool algorithms are:

del.icio.us / delicious.com 's: ranked by upvotes in the last hour. I like the elegance of this solution a lot :)

reddit's "best" feature: compares the rates at which comments are upvoted to generate the probability that a comment is a good one, which is how posts with fewer votes can be ranked better than posts with more votes. It works well, in my opinion.