News.YC faster (plus new traffic stats) | Hacker News Reader

News.YC faster (plus new traffic stats)

ycombinator.com

117 pointspg82 comments

But we're hoping that, as in past influxes, the new arrivals will with some prodding from the existing inhabitants learn the local customs.

I'm absolutely smitten with this form of social hacking. To me, this seems to be the most revolutionary part of the site. That controlling "free" speech is not an impediment to open speech and that community members can be taught, through participation, to behave according to some simple constraints.

My apologies if this is an inane realization. It just doesn't seem to be said often enough in on-line communities. Lassez faire doesn't work and it's possible to also encourage the worst, trollish, impulses. This is the first clearly positive example I've seen. If there are others please let me know!

MetaFilter also works this way. Self-policing is a big part of the community [1][2].

There is a even a whole section of the site devoted to discussing internal stuff (called appropriately MetaTalk [3]).

Sometimes I miss something like this on HN - a separate space for meta stuff.

BTW MetaFilter is the only other online community I'm aware of that has quality on the level of HN (despite being even larger - over 56,000 registered people).

[1] http://mssv.net/wiki.cgi?SelfPolicing

[2] http://en.wikipedia.org/wiki/MetaFilter#Moderation

[3] http://metatalk.metafilter.com

Actually, I kind of like that HN is self-meta (i.e. the ability to ask questions as well as post links). There were a few good discussions about how to make a good post a while back: http://news.ycombinator.com/item?id=419539

I think his point was that meta does not have its own section, and that because of this some meta comments are voted down (despite validity, good points, etc).

Metafilter seems pretty cool, but it could desperately use some formatting to help its readability. Basically copy the HN/reddit styling to differentiate posts and comments from each other.

Most Metafilter commenters would strongly disagree, and do when the subject arises. Threading is especially frowned upon.

> open speech and that community members can be taught, through participation, to behave according to some simple constraints

That's the bread and butter of most open source communities, and part of why they tend to work: open source, in my opinion, is primarily a social phenomenon and the forces for social cohesion, whether in terms of coding standards or ideologies quickly produces communities that work well together online.

yeah, group think ROCKS!!

nah, I know what you mean ;), it is mostly good for now.

prove that with sufficient caching you can serve arbitrarily large numbers of requests with arbitrarily slow languages

I love the way that's worded, but I do have a serious question: how do you do the caching in a way that keeps content fresh? For example, you couldn't just cache this page with the comments because then I'd think mine didn't show up. Or do you add it to a comments cache in memcached or some similar mechanism as well as saving it to the database? Or simply expire the cache on save assuming that there will be more reads than writes?

It's always seemed hard to do good caching for a site whose content is constantly changing as people comment and vote on items and it'd be nice to see how complex the model used here was. The latest source I know of is at http://ycombinator.com/arc/arc2.tar. Will there be an update with new source?

Oh, and congratulations on an awesome site!

"...with sufficient caching you can serve arbitrarily large numbers of requests with arbitrarily slow languages."

Only if the dataset is small relative to the traffic rate, and if the data mutation rate is relatively small to the size of the dataset. There are plenty of systems that do not have these properties.

I agree the speed of the language does not matter as much as people used to think... but the shape of the data matters a lot more than people think.

The shape of the data matters particularly when you want to scale beyond that initial single server, which is especially relevant if performance is weak.

Maybe it would be helpful if there were a way to see a user's 100 highest rated comments. I'm thinking it would help both in socializing the influx of new users, and also in getting an idea of what a given person like.

Good idea.

Don't downmod this guy(gal) in to oblivion just because (s)he didn't say too much or you don't agree... Especially in this thread. This should be reserved for trolls. I think a nice 0 or -1 (to discourage pointless comments) would be fine but (s)he's already at -3 when this is still a pretty new submission. Don't hop on the bandwagon just because.

Replying to comment with "Good Idea." serves no purpose.

That user's account is nearly 2 years old, they know better than to post that.

Pointless comments should be discouraged just as much as trolling should be.

Currently, just clicking on a username and then clicking on the comments link in the user profile gives me a pretty good sense of what that user is like.

My highest rated comment so far occurred yesterday, when I got 30 points for saying something reasonably good early on in a popular post. I think other people agreed with what I said, and so (appropriately) just upvoted the comment rather than repeating it.

If your proposed feature just ended up displaying lots of comments like these, it might not be that great.

A somewhat related idea might be to let longstanding/high karma users add a 'top N favorite comments' list to their profile. When you looked at a user's profile, you could click through to see their favorites, and the comments by them favorited by others. (Users could add their own comments to their favorites list, if they wanted to.)

I can think of several users right now whose favorites would be interesting to me.

This is why some indication of a user's median karma per comment would be at least as insightful as his or her total.

SearchYC.com can do some sorting of user's comments.

e.g. http://searchyc.com/user/Alex3917

Yikes, link appears to be broken!

Long time reader, first time poster. Just wanted to say that the quality of submissions and comments is what always keeps me coming back to HN, and it's great to see active interest in maintaining the quality.

You'd never hear something like "Growth can't keep going at this rate forever without ruining the site, though. Between those two alternatives, we prefer growth to slow down. We hope it will happen naturally—that we'll simply run out of new people the site appeals to." coming from a profit-driven site

would be interesting to see what kind of hardware HN is hosted on

Also, somewhat related to this, does anyone know if newmogul.com runs on the same code or does it just look identical?

probably the same code - it's open source.

Ahh! I didn't know that. Thanks!

where? i can't seem to find it

http://www.arclanguage.org/install

Now that I am learning PHP to do it you tell me...

I'm writing a PHP interpeter in Java to render COBOL, which'll execute on COBOL on COGS.

I forgot to mention the top level will all be written in XML, so it can be used on the semantic web to provide a service oriented architecture.

But, next week, I'll be starting a project to design a middleware system based on a LISP repl. The foundation will be written in C and assembly. The dynamic portion will follow Steele's CL standard, although it'll be able to compile select portions to machine code during runtime, if necessary.

Since LISP is equivalent to the parse tree, it'll provide the RPC IDL so programs written in Haskell can communicate with programs written in Erlang, all seamlessly across a cluster or grid with a parallelization mechanism based on the STM model.

Joke explained:

The original comment made a joke that mentioned PHP. It got downvoted. I thought I would make a joke myself based on the idea that it got downvoted because it mentioned PHP.

Then, something went horribly wrong.

umm why are all mentions of PHP being downvoted? i might not understand completely, but that seems like a puerile thing to do.

It's not the mention of PHP, it's the tone of the comment.

Tone is hard to interpret without a tone.

I'd say yes. The HTML is identical right down to the missing XML declaration.

Leaving off the XML declaration is common because having it there causes IE6 to run in quirks mode. Although it's probably safe to say HN runs in quirks mode on all browsers.

This is true for Safari, Firefox, Opera and Internet Explorer browsers. Not providing a doctype whilst rendering text/html will make the browser render in quirksmode.

Last I heard, a single box.

A single box can range from an old 286 to something that powers plenty of fish.

Does anyone else feel like HN is actually ahead of the zeitgeist than other sites like Reddit and Digg? Obviously the focus is closer with technology/entrepreneur articles but I keep finding stories I've already seen on HN hitting Reddit and Digg up to a day later.

HN is beating some of the specialty discussion sites I like best in timeliness for certain specialized topics, so it appears that people who find cool links are defaulting to posting to HN first (which is also becoming my default behavior).

> "[W]ith sufficient caching you can write popular apps in arbitrarily slow languages."

Somehow I doubt this pronouncement will stem the tide of statements along the lines of "x is written in a slow language and therefore won't scale", although they have been mercifully less frequent around here of late.

On a slow satellite connection I definitely noticed the new zippiness. No other site I visit loads as quickly - not even a google search.

I'm curious about a satellite connection. I spend a lot of time in southern Spain where we don't have landlines. We do have GSM so we get _slow_ internet.

Can you shed some light on satellite internet? Who are you hosted with? Hows the speed & reliability?

We're with HughesNet. It's bad (200 kbps) at peak times, better at moderate volume (500 kbps), and pretty good at off-hours - 5am (1.5 mbps). We pay about $80 month but it's our only choice besides dial-up. It seems they now have faster connections if you're willing to pay more (http://go.gethughesnet.com/plans.cfm)

Reliability is really good. It's hardly ever down, even during major snowstorms (just have to clean off the dish). I'd say we lose connectivity maybe a few hours a month.

The worst part is the low daily bandwidth limits (425 MB in 24 hour periods) and then they throttle the service (<20 kbps) until you're into the next 24 hour period. They don't tell you when you're getting close. Since we started using a bandwidth tracker to monitor ourselves we haven't had any problems but the first few months were painful and frustrating. They do have three non-monitored hours (4am to 7am local time) so you can schedule downloads accordingly.

Not sure if they're international though. Be careful of on-line reviews. They're almost all negative because I don't think folks realize the technology is always going to seem more limited than DSL or cable. Coming from either of those (or a T1 etc.), you just have to reset your expectations.

thanks a lot - really handy information. I'll look for some global providers. Doesn't seem unreasonably priced, either.

Good luck with your venture/startup by the way - sounds fascinating.

Glad to help.

Thanks!

There are few providers of Satellite Internet here in Australia - in fact if you're in an area that doesn't get DSL coverage the government subsidies it.

Probably work keeping in mind there are two main varieties - one where the uplink is over a standard modem connection and one where the uplink is over the sat link. The former tends to be a lot cheaper for obvious reasons, but means you need a landline.

Telefonica does satellite inet access in Spain. I remember prices to be quite high some years ago. You could check their site and see how they fare now.

great lead, will check it out now - thanks!

1) Any idea how many page views per month?

2) Is all the HN data (comments, submissions, users) backed up and stored off the machine it's running on?

1) Looking at the graph from the article:

http://ycombinator.com/images/traffic-8dec08.png

For December the average page views per day averages about 200K => 6M page views.

This is great.

Are the events that brought in the largest spikes in traffic known? Do page views correlate to new accounts?

I belive the huge spike was techcrunch

hmm, that spike was before techcrunch covered YC (they covered it in December) ... no idea, then :)

I'm wondering whether the huge page view spikes that have no corresponding unique IP spike are actually caused by crawler traffic.

Usually crawler traffic.

http://news.ycombinator.com/item?id=390965

The largest spike looks like decision day. (That's the day YC sends out emails notifying the people they'll fund.)

awesome.

News.YC is still my primary source for most news, it's an amazing community.

Two months ago I was worried that with the Obama surge things would break apart and collapse and become generic as a site like Reddit's become. To my surprise, it's managed to hold together incredibly well. And I love it: this is absolutely the best community I know of on the Internet.

"it" didn't magically hold together. that was the work of the editors! if you read the site with showdead turned on in your profile, you'll see that a record number of users and submissions are getting killed.

just to be clear, that's not me screaming censorship. it's rare for me to see a dead comment or submission that i wouldn't have killed myself, if i had the power to do it.

Yep! I flag a lot of stuff in the new section. I'm just always surprised to see how effective that is at pruning the site.

Besides the excellent moderation, another huge contributor is that instead of being a generic medium for posting articles and discussion, like slashdot, HN is founded on pg, in a sense. His ethos is the site's ethos.

yes - but without the cult-like persona following that often arises in situations like these.

Which both he and us probably thinks is good.

without the cult-like persona following

To be fair, maybe a little cult-like persona following...

Are you trying to get excommunicated?

I've been reading News.YC for last few months and it's now my primary source of IT news/discussion. While I restrain myself from posting sometime because I'm not perfectly fluent in english, I have deep admiration for the community and it's insight on the "hacker" world.

Thanks alot!

I think that around here it's more about what you have to say than about being able to do so in perfect english. Besides it's a great way to learn...

So if you have something to add the conversation do so. Even if it isn't perfect english :-)

In fact, you're likely to receive a friendly correction on severe grammar or semantic errors.

yes - with an emphasis on friendly.

Thanks, it's easier to participate when you know that your interlocutors are able to understand that just because you don't master their language perfectly do not mean that your ideas are any less valuable.

I agree, and what's more, It feels "livelier" to me lately, which could be the feel of the growing population. I'm not worried though, the discussions have continued to be excellent.

So we just finished a new round of optimizations as part of our ongoing quest to prove that with sufficient caching you can .....

PG, can you please list the various optimizations? Thanks!

Oddly, just now my access failed to HN using either of two different browsers, several times, even though other sites that occasionally have traffic hiccups responded just fine.

...and there has been a slight uptick in comments that are insulting or inane.

Is there a way to measure this? I know someone mentioned a filter for Youtube comments before...

I'd like to see increased moderation to balance the new users.

If you watch the site with showdead turned on, you'll notice that moderation has increased as the userbase has increased. There are more dead items than ever, but such is life when trying to maintain quality.

So in short, you are seeing it.

One feature I would love news.yc is a page to list only the Ask HN submissions... I like reading articles and seeing the reactions to articles but the Ask HN submission usually end up having higher quality content on average IMHO.

http://ask.searchyc.com

http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive

Courtesy of chengmi, alaskamiller and epi0Bauqu

I realized today that caching breaks one of Lisp's cardinal rules (as quoted from OnLisp):

The convention in Lisp seems to be that an invocation owns objects it receives as return values, but not objects passed to it as arguments.

How do experienced Lispers deal with this? Just being careful?

I don't know if this is because of the new fix, but for me submission pages won't load. Any user's submission page just freezes. Safari 4, OS X 10.5.6.

Edit: never mind, working now. Possibly an Internet malfunction.

Why is the alexa rank so low for the kind of traffic this site gets?

Reply on news.ycombinator.com