Where is this documented? Not trying to be snippy - I have no idea what powers the moderators have in this realm. Maybe HN could benefit from some lobste.rs-esque moderation transparency.
I like it this way. They have excellent taste. The community has repeatedly proven it cannot be trusted to upvote intellectually gratifying content. pg's original HN code contained a flag for "rally", meaning the story is a rallying cry, and is therefore penalized because it's so easy to mindlessly upvote it.
The results speak for themselves: HN is now almost a top-1000 site by traffic.
They had to be phenomenal. I don't think many people appreciate just how difficult this was to pull off, or how much they work their asses off every single day. If they ignored HN for even a week, it would deteriorate noticeably.
Yet technology is at least as important. The fact that our upvotes do matter makes the situation unique -- although the mods can determine the relative weights of stories, we can still push a story upwards if enough of us are interested, and the story fits the site's original goal of gratifying intellectual curiosity.
There is no company anywhere in the world like this. At this point, HN practically defines our culture. When news stories say things like "so-and-so story about <technology> has been making the rounds on social media," at least some of them are talking about HN now. And your coworkers probably hear about the tech they use at least partly from HN.
The sole reason they're in this position is because they deserve to be. I hope they stay selective. As to being secretive, we should let them choose. If they want to reveal, fine. If not, fine. They've earned it. Even still, they've been remarkably transparent.
There are probably a bunch of people behind the scenes, and they deserve lots of credit too. But there is something incredible about being able to post a dozen moderation comments every day for years without ever being so mistaken as to provoke community backlash. It would be really easy for them to offer an offhand opinion, or just make a bad call, and suddenly have people up in arms about it.
They're also some of the most humble people. Both of them have done some really amazing work and have cool hobbies. But I've sort of had to pry that info out of them over years.
http://github.com/gruseom/numen - Dan's project. JS REPL for emacs. It took another four years for anyone to even come close (https://github.com/NicolasPetton/Indium) as far as I know.
https://github.com/sctb/lumen - Scott's project. A self-hosted Lisp that compiles and runs in both JS and Lua. The whole codebase recompiles in about one second flat when using LuaJIT.
But like, that's maybe 5% of the cool stuff they've done. There's just no outward sign -- they don't bother with blog posts or any kind of self-promotion.
So yes, all hail the kings of lisp.
I have some qualms about some systematic things, but it's a broad philosophical dispute and not the running of HN. In governance - there is no way to 'win' anyway.
I wish every leader had to be a forum moderator of very different groups of people, but we seem to select for whatever the opposite of that is no matter the political predilection.
Indeed this is cool.
To compile on BSD, I used
gcc -static -std=c89 -Wall -pedantic -o woe woe.c -lmEdit: fixed, thanks!
The freedom to take other people's private belongings and repurpose them to your own ends is no real kind of freedom at all. But that's ultimately what you're yearning for when you allude to the Empire of Napoleon.
Diff that vs HN circa 2008: https://github.com/evanrmurphy/SweetScript/blob/master/arc3....
Lamernews seems like just another HN clone, but back when it was launched it was a serious attempt at being a competitor. They probably came closest to pulling away a significant chunk of HN. I started to use it.
Then I ran into missing features. Little quality of life stuff. I forget what they were, but the whole site felt noticeably harder to use. So I stopped using it.
It's so tempting to think "HN is so simple. Just throw Rails at the problem and you can wipe the floor with them." But Lamernews was 2,964 lines of Ruby, and only managed to contain about 60% of the features. news.arc is 2,616 lines. And I don't care what anybody says: if you sat down and forced yourself to read that code for a week straight, you'd find it is actually quite readable. It's terse because it's for getting shit done.
And it all worked! No data loss, no massive security problem, and the last time HN was down for >24h was because pg manually edited the database and accidentally created a circular comment chain.
One way to look at the situation is that pg's original vision for arc failed. Arc was supposed to be what Clojure turned out to be. But even though HN alone is the one significant Arc project, what a project! The world was made materially better thanks to it. And I don't think they could've pulled off HN in some other language. There was too much work for one person to do to be distracted by the thousands of details a modern webdev needs to stay on top of.
I think it's possible to do the same thing for modern webdev. Arc's goal was to be a reliable foundation for building webapps. It comes with a user+admin system out of the box, for example. Every Arc app does.
I've been playing around with hooking up AngularJS to it, because unlike React or Angular, AngularJS can operate purely in HTML. You normally never want to do that -- usually you want to set up controllers and such. But when you control the language, you end up being able to give it the Babel treatment: the output looks complex, but you get all the features of a modern frontend site. Initial experiments were quite promising.
And from there, it's easy to just emit JSX instead. React? Angular? Sure, whatever. It's treated as compiler output.
(Indirect sourcemaps are funny... JSX compiles to minified JS, whose sourcemap points back to the JSX. But if Arc spits out JSX, you'll want a sourcemap from JSX -> Arc as well. It's sourcemaps all the way down.)
EDIT: Actually, you may be right. They already forward data to Firebase, which means they could just store the data in Firebase directly. Firebase is pretty great, so it's possible they did this.
One bit of supporting evidence: there seems to be no startup time. Loading GBs of data from disk takes a little while, so restarting HN would be painful. We'd notice it as ~30 seconds of downtime. Especially now that HN has grown so large. I haven't noticed this in a couple years (but I'm not omnipresent).
So it could go either way, and there are a lot of advantages to the Firebase approach.
I guess I take back everything I said. But only because a lightweight alternative exists, not because flat files are inherently bad.
Here's how it works. Everything is loaded into memory at startup. You can think of it like a bigass JSON tree. Anytime you change the tree, write the whole thing out to disk. Since all the data is loaded from file "foo", and you write out to "foo.tmp" followed by `mv foo.tmp foo`, there is no risk of data loss, ever.
So what about sharding? Well, yeah, that's a problem. But it's not a problem that HN had to deal with. You can see how effective it was to throw a single 64GB RAM monster at the problem rather than a cluster of servers. The moment they hooked up Cloudflare to HN, they got all the same benefits a big cluster would provide anyway.
People can be prejudiced against this simplicity all they want, but it's hard to argue with results. You don't even need to know SQL to start using it.
"But migrations --" well, Arc makes it super easy to define a "template", which is just a set of fields, and each field is whatever you want them to be.
https://github.com/evanrmurphy/SweetScript/blob/6b4ed26acc9c...
(deftem item
id nil
type nil
by nil
ip nil
time (seconds)
url nil
title nil
text nil
votes nil ; elts each (time ip user type score)
score 0
sockvotes 0
flags nil
dead nil
deleted nil
parts nil
parent nil
kids nil
keys nil)
It's sort of similar to mongoose record definitions.
But look, HN is proof positive that this system can work and can scale to a top-1000 website. 99% of webapps won't even come close to those performance requirements. And when you do approach them, you can find ways to make it scale.
My main focus has been updating Arc to run on JS, and to add keyword arguments. That way you'll be able to use the whole JS ecosystem too.
That's... just a struct or a map in any other language. Convenient but not necessarily a point in Arc's favor per se.
>So yes, theoretically if you wanted to rename "kids" to "children", you're gonna have to process all the old data. But 1. then don't do that, and 2. even still, it's not hard to do that anyway.
It's still an argument against using flat-files. Updating a site with a proper database isn't typically tied to disk read/write speeds - at best renaming "kids" to "children" means changing a field in a model and maybe a template - and that's a fixed amount of work regardless of the number of records you have.
>My main focus has been updating Arc to run on JS, and to add keyword arguments. That way you'll be able to use the whole JS ecosystem too.
You mean Node/NPM on the backend? That might be interesting. I'm not familiar enough with Arc to know (I've only played with Anarki a bit, but I have too many random projects on my plate as it is) but does it even have dependency management?
Jokes aside, I refuse to let Arc die. There were genuinely good ideas embedded in it, amidst the rough edges. People just never forced themselves to use it. It's like if Vim was invented but had no documentation, so no one ever realized it's powerful if you embrace it.
But yeah, I imagine my version will have a different name and a shiny website. Being a decade out of date does matter, especially for hiring purposes.
As far as I can tell, the arc forums site is still up and running, and a lot of people seem to be focused on Anarki[0].
Forums written in other languages do as well - and any decent framework can just include user management as a dependency. To me it doesn't seem like an unqualified good idea for Arc to bake all of that into the language or the default app.
Bam, you get CRUD for free, you get special superpowers if you're logged in as an admin, and you can put in access controls so that other people can log in as other types of users and see a subset of the data. I dunno, it's pretty great. You even get a REPL at /repl in case you need to hop on and run some code on production. (If something seems amiss, rather than bother with grepping logfiles, you'd head to /repl and start asking questions about the current state of the app. This tends to be a lot faster.)
Just depends what you're looking for.
I like Lobsters a lot, and I like a lot about what they do differently than HN, but we shouldn't pretend that it's something it isn't.
There are reasons people continue to frequent HN as a news aggregator and comment on the site: it has some uses and benefits to them. 'sillysaurus3 does list a number of reasons supporting their position. If they're expressing a bit more joy and appreciation for your taste, dismissing that as "slobbering" seems quite harsh and unnecessary. Phrasing it as "pro-moderation" in an arguably pejorative manner likewise derails. A quality site, one I find personally valuable, requires moderation. If I'm in favor of a particular site and value its moderation efforts, I'm going to be "pro-moderation". If I actually find it valuable, enjoyable, and a benefit, I'm going to be appreciative, and maybe at times effusive. Have I lost objectivity in such a case? I'd like to think we're more human than that.
I agree it (along with other moderation behaviors described in the article) should be documented more explicitly in the guidelines/FAQ
When google released its page-rank algorithm it became a target for manipulation and entire forums got wrecked (until nofollow) with the wave of consequences it created.
I think the threat of user manipulation will always outweigh that of mod manipulation. Worth considering.
There's no master plan behind it all, but each of these ways of handling policy has a purpose. Describing evolving policy in comments allows the moderators to be responsive and flexible without overrunning the site with metadiscussion like they would if every new decision they made was accompanied by a formal change in the guidelines. Relating policy in email discussion gives them a chance to explain it in context and to take feedback from the people most impacted or interested in that policy --- we all know votes on this site wouldn't generate that set of users.
The stuff they don't make public is generally stuff we don't want made public. We forget that there are just a couple (I think literally) people running this site, and they're up against waves of people trying to create schemes to sell front page rank to spammers. They've got a sort of equilibrium that gives them a chance --- something Reddit seems to be failing at despite a much large team --- while still having the time to type out individualized contextualized explanations for all their routine moderating actions on threads (seriously, mods, figure out TextExpander or build something like it into your moderating tools).
It strikes me it's always worth holding that this is a subdomain of YCombinator and presumably is used to further that corporation, its controllers ideals, and its associates interests over others who might seek influence here.
There are things (e.g. changed titles) were I wish their specific action would be visible to avoid confusion (they often leave comments, but not always and they are easy to miss). That'd be more useful than a list entry somewhere "mods can change submission titles".
People often don't seem to know about flags and vouch, these tools for users could be better explained somewhere. (Not that it will stop those crying "moderators censoring $thing!" every time a controversial topic is flagged)