There definitely needs to be a more fluid/integrated means to discover new stories. The home page could be split down the middle, showing new stories to the right, or there could be a more feature-rich (and visible/accessible/usable/seductive/natural/etc) "upvote" page that allows searching, browsing, and filtering through submissions to discover ones to vote on. There could also be a slight weighting of votes (though not much) to create a bias toward more reliable (or longstanding) voters, or to infer which submissions the currently viewing user is likely to upvote. The bias likely has to be slight, as one part of the game is to show the global/absolute top stories, while also getting in stories the specific user is likely to find "upvote worthy."
Also, randomly surfacing new submissions to get a sampling of up or down votes seems essential. This would allow a greater spread/certainty when a submission does make it to the front page, and would also be fairer to promising submissions (they won't die prematurely due to lack of opportunity to be upvoted). It would also minimize gaming of the system and the relevance of "time of day."
Also, the gravity concept is good (so stories eventually die), but it's a bit too absolute/aggressive.
There should also be normalization by total number of impressions and up/down votes, such that raw volume doesn't overwhelm quality with lower volume. That is, even if confidence intervals are used (which I don't really like), then up/down votes are an estimate of overall up/down vote percentages, which is what's relevant (the submissions most likely to be upvoted (meaning "best") are what should be shown first). Also, maybe volume matters somewhat (many votes implying a lot of interest), but it's better to separate this out and factor it in independently.
For example, each signed in user that clicks on a link has a probability/likelihood they'll vote, so that could convert their impressions into an expectation which could then be compared to reality (how many upvotes are actually recorded). The higher the upvotes (compared to what was expected), the better the submission. This can then be combined with the other scoring methods (and slight personalization-- maybe a certain percentage of front page entries are personalized and the rest based on absolute rankings, rather than having everything be personalized, or breaking them up into separate sections) to stabilize, refine, and make more accurate submission ranking/sorting.
There isn't a problem with a downvote button. And there isn't even a problem with it being abused (it being abused negatively affecting rankings is a sign the ranking algorithm is poor). The problem is with downvotes not being properly factored in (and weighed up or down depending on the source of the vote, and the user viewing the resulting front page articles). In fact, adding a downvote button would make it easier to rank new submissions (and it's also straightforward to filter (or normalize) out abusers of upvotes or downvotes).
Also, maybe passive actions could be factored in (or things could be added that would serve as good "signals" for "front page worthiness") to increase the overall number of votes, such that a handful of users don't control what makes it to the front page and to allow each article score to be more reliable/accurate.