PG: What's your current take on reddit's comment system?
reddit.com
While reddit's comment system is obviously an improvement over Slashdot's, I'm increasingly convinced that the community is more important than the tool.
reddit.com
1) Small groups have higher numbers of core users
2) Private and semi-public groups have higher numbers of core users than public groups
3) People in private groups remain in the core for longer
4) A user who is in the core of one group is more likely to become a member of the core when joining another group
5) New members who eventually become part of the core are more likely to receive messages from other users who have been long-time core users
6) New members who eventually become light users are more likely to receive messages from other light users, if they receive replies at all.
This suggests some strategies for building a strong group of core users to transmit social norms. For example, having an idea of who the core users are in other similar communities and making a special effort to reach out to them if they visit your site.
Unhappily I lost the original paper and all I have are my notes, so I don't know exactly how they defined the core users offhand.
You can email me if you like.
I've developed this hobby of sneaking into academic conferences. It works best if when everyone else is wearing a suit and tie, you show up in shorts and a t-shirt. That way everyone just assumes that you're friends with one of the speakers, or else that you're Sergei Brin. :-)
http://research.yahoo.com/publication/structure_and_evolutio...
I probably won't have time to read it until this evening, but will post then.
What would be equally interesting as these findings, are the premises. What service, what content, what medium, what purpose, and extremely importantly, how presented.
It actually sucks me into the troll MORE when they get such special highlighting. I should say I do like how top level troll posts get put at the bottom for reddit.
Only submitted articles do.
I have no idea how you got any different idea into your head, but it was not through observation of reality.
My solution was to store a comment thread in display order in the database with hinting as to how deep in the heirarchy the comment is. Optimizes for output, although with some cost every time something gets posted. Luckily, for most forums, views outnumber posts by hundreds of times.
The one issue that system has is that if you're doing a lot of reordering, let's say from quality scores. In that case you'll basically have to remake the thread after every post and after vote that makes a difference. Not recommended...
[edit] especially when the comments get reordered on votes.
edit on the edit: scratch that.
Y'know how in a typical CS datastructures course you'll have to build a heap, and then reimplement with an array? A heap is conceptually a balanced binary tree where every leaf is filled, while there's nothing tree-like about an array. However, your CS profs were trying to get across that any binary tree can be stored as an array, with the root in position 0, its leaves in positions 1 and 2, their leaves in positions 3, 4, 5, 6, etc. If a node doesn't have a leaf, that position in the array is empty.
Normally this is very inefficient - a totally-unbalanced (linked list) binary tree of length 20 would require a 1MB array to hold it. However, heaps are always perfectly balanced, so an array really is the most efficient implementation for them.
By restricting users to a certain number of replies to each comment (65,536), I'm effectively constraining an arbitrary tree structure to a 65,536-ary tree. Which can then be represented by an array. If held in memory, it'd be a very sparse and inefficient array.
But databases are excellent at storing sparse arrays. If I don't store a potential key, it doesn't take up any space in the DB. And with a good indexing system, I can reference any element in constant time (well, O(logN), but the log base for B-trees is so huge that it might as well be constants).
So, I'm really just applying some of the meta-principles of CS coursework to basic data structures. (Actually, I got the idea off a Slashdot posting and had to figure very little of this out on my own, but it's still really cool.)
Incidentally, there's apparently a deep connection between number systems and tree structures. You can read more about it in Chris Okasaki's Purely Functional Data Structures. Apparently, every number system corresponds to a data structure. For example, Peano arithmetic = linked lists, binary numbers = binary trees, fibonacci numbers = binomial heaps, etc. There's a short presentation on it here: http://www.informatik.uni-bonn.de/~ralf/talks/BCTCS.pdf, and I'd strongly recommend the book.
"Premature → evil."
Edit: See Ruby/Lisp for one such already built interpreter.
comments = dbh.query('SELECT * FROM comments WHERE news_id = $newsid')
response_tree = {}
for comment in comments:
comment.children = []
response_tree[comment.id] = comment
for comment in comments:
response_tree[comment.parent_id].children.append(comment)
However, I'm not sure how else one would want to model things. Must think more...
Many filesystems have trouble with large numbers of files in a single directory, as you'd get with eg. pickle tree to a file per item.
As for storing the pickled output:
dbh.execute('UPDATE articles SET comments = $comments WHERE news_id = $id',
comments=pickle.dumps(article.comments))
Reordering can be done just by looking at all records whose IDs contain the parent as a prefix and have a length 2 greater than the parent, and then renumbering them. This should be computationally feasible in most cases.
This does limit users to 65,536 replies to a single comment, and a maximum comment nesting depth of 127. Based on my experience with some very active LiveJournal threads, I considered these to be acceptable limitations (LJ limits comments to 5000/post anyway). A nesting depth of 127 would be nearly 2400 pixels over, so it's not like it'd all fit on one screen anyway.
Also, each comment should have a newsid field that points to the original newstopic. The left-most posts would have the parentid and newsid that were the same (or parentid can be zero, depending on how you do it). But any replies would have the parentid to the parent comment, but the newsid would still point to the newstopic. Then you just run an SQL query to find all comments with the newsid for that newstopic. You won't have to join any tables. After that, with everything in memory, you can just sort the way it has to be. Finally, you can cache the page in memory or a file in case the page doesn't change for the next set of visitors.
To simplify the recursive function (using data in memory, not in the database), you could also have another field, called indent, that stores the level in the comments.
You could find all the comments with one SQL statement (where newsid = news.id), then go through each level of each comment, and sort it that way.
You could even make the parentid field a double, so you would store 123.4 where 123 would be the parent and 4 would be the indentation level.
I guess the difference is, do you want to store everything in one field or have different fields for everything?
I think this gives the original poster many options to think about. To me, a recursive function would be easier to write than coming up with ways to combine multiple fields into one using MySQL.
I think the order of optimization is: 1) Optimize so you can use a cached result if it exists 2) If the cache is out of the date, try to make just one SQL call 3) don't use table joins 4) reorder the data as needed in memory, then save it to a cache
As far as the single SQL call goes, I don't think it matters whether you have 5 fields or 30 in the actual table as long as you only request the fields you need, but I could be wrong.
By the way, do you have any other interesting examples?
To be fair to the reddits, it wasn't their fault that the discussion on reddit sank down toward that of digg and slashdot. They have really strong beliefs against censorship. They were never going to jump into a discussion and tell people to stop being jerks. But empirically it looks as if you may have to.
The optimistic way of phrasing this is: if you have good customs and existing users enforce them, you can probably survive the influx of 14 year olds when it comes.
http://www.shirky.com/writings/group_enemy.html http://shirky.com/writings/broadcast_and_community.html
I agree with most of his points. Better technology can not substitute for better people, although I observe a strong correlation between mental acuity and humility and, hence, 133t hacker skills.
One of the unfortunate things about communities (reddit included) is that they tend to gravitate towards a certain conformity of opinion which then rewards people who parrot the standard opinions and stifles innovation.
I was reading a blog post recently on how someone creeped to the top of Y-Combinator karma by observing karma-award patterns and posting articles which he thought people would award karma for.
This, obviously, is an effective way of winning prestige, but prestige has rarely been a very good correlate for anything. In fact, as Pg stated in his talk at Rails Conf, even at the individual level an increase in prestige frequently has a negative impact on innovation.
Redditors have been observing the same pattern in their own community. As more people have become aware of the community (the prestige has increased), the quality of the postings and karma system has declined to the point where inane 'Bush resembles Chimpanzee'-type postings dominate the main page.
There is ultimately only one solution to this problem - the creation of walls. The last online community I participated in went through a similar phase of evolution. The content was progressively diluted until a number of core members went and created their own community. This new forum/community now has a application for membership which requires applicants demonstrate writing proficiency prior to entry.
Tie-ins to current political machinations could be articulated, but seem unnecessary at time.now
The same algorithm could work for postings. If I want more Ron Paul, peace be upon Him, I can privately champion those authors who celebrate his news items. You may have had enough of the distinguished doctor-representative from Texas, so you could exercise your prerogative to ignore (or artificially down-mod, just for you) submitters whom you view are overly sympathetic to Him.
Further thoughts on reddit's comment system...
Problem: Logical fallacies largely go unchecked. Just today, someone replying to my comment pulled a tautology so I reflexively linked to the appropriate Wikipedia page. As mentioned, ad hominem is a more popular sin and the list goes on. What if, when I clicked on the "Report" link, I could select from several logical fallacies? In this way, if enough others agreed with me, we could get an official-looking stamp on the offending comment--corresponding to what the community felt was the logical fallacy (e.g., straw man, false analogy, non sequitur).
Problem: Today, if I submit a comment and I down-mod all other comments I increase my comment's visibility. I want my comment to have visibility (that is why I wrote it) and I suspect other redditors may employ the cheat. So I face an ethical dilemma. We can talk about real-life karma, but why tease commenters to begin with? Solution: If you comment you can't vote in the thread.
Problem: I imagine a lot of redditors, like me, sort by top->flat and I suspect most of the remaining redditors sort by top->nested. So if I comment, I significantly increase my visibility odds if I reply to the front-runner comment. To the extent I care that people will read what I write, I will tend to favor the popular comments--even if they are not as worthy of a response as those nearer the bottom.
Pg speaks above about customs, but the problem with customers have to both (1) be educated about site customs (2) want to be educated about site customs. I think with any influx of 14 year olds of sufficient you will find both 1 & 2 problematic.
Now, personally, I think there is a significant difference between a news aggregation site and a community site. In a community site persons actually develop relationships and relate to each other on this basis. I have no seen that happen much on reddit or here. One reason for this is the speed of discussion - most of the best online discussions I have been involved with have taken days or weeks to fully develop. In any news-blog-digg driven site the time frame is closer to 24 hrs.
What I would favor is a multi-tiered system. For something like YC News there could be something like three tiers: (1) Old school hackers with experience in startups (2) Less experienced coders (3) Whoever wants to participate. Each conversation could be assigned a tier, and people of that tier and above tiers could participate. For example, a tier 3 discussion would accommodate people of tier 1,2 &3, a tier 2 discussion could accommodate tiers 1&2.
In many ways this model is what we already do. We have low-tier sites like digg which we may check, we come to places like this for a free form discussion mostly among people of type 2, and type 1 discussions are mainly engaged in via email. So in some ways the only innovation I propose is that some of these type 1 discussions be viewable to the public, so we can see what cool lisp hackers talk about in private (and solicit them for advice).
Response to other reddit comment comments:
Fallacy - I don't think the fallacy thing will work simply because (as with the site customs described above) one has to be educated as to what any given fallacy means.
Ethical dilemma - funny I never thought of doing that, although I think your solution would work.
Popular comments - yes this is a big problem; I have noticed it.
Nevertheless, based on this perspective, I suspect the tiered system would not work, simply because we don't think in tiers in real life. Everything is graded in subjective degrees. One day your Lisp email list would evolve into a nonsensical banter (somebody starts posting after getting drunk). One day a couple geniuses post on digg and write a bestseller novel on the comment system within 24 hours. Perhaps unlikely, but not impossible. There are zero barriers in real life that let this happen; when it needs to happen, it will. But online, the system becomes the barrier.
So instead of a multi-tier system, I believe the best system is bound to be an aribtrarily-tiered system. How might this work? Consider yomama on reddit, who posts yo mama jokes on every thread. Troll yes, but also a clown. If there is a clown thread, yomama gets the crown (sorry). Thing is, there are "tiers" in the sense that you only have upmod and downmod. And really, what do those mean? A tier of good, a tier of bad? Predefined categories will never be enough.
If there are disagreeing voices, I'd love to hear.
Now I'm not arguing that we should necessarily replicate real-world institutions in online form, but one should at least recognize that real-world institutions (like universities) provide various measures which enforce content quality.
As for approximating real discussion, I'm not sure exactly what you mean. There are many different types of real discussion, from idle conversation about sports to academic journals about the poetry of W.B. Yeats. Although certain similar principles apply, the way in which you would model either of these would have to be different.
I'm afraid I also don't understand what you mean by an arbitrarily-tiered system. Are you arguing in favor of the reddit model?
The examples you give about discussion are all "real discussion." What I'm saying is that I believe the ideal discussion system would accomodate all these discussions. Poetry, sports, academic journals, etc. If not now, eventually it must.
While sometimes it makes sense to nicheify, like separating a fashion-specific website from a tech-specific one, the material isn't always mutually exclusive, and so lots of information is lost. There should be channels for different discussions to connect to each other. Wikipedia is one example of how it can be done.
I guess arbitrarily-tiered system is a bad phrase. I just mean no explicit tiers, like, you have a bit set for mod status, a bit set for troll status, etc. Everybody will interpret authority, or hierarchy, differently, so just leave it as-is, at least with subjectively assessed content, like puns, politically incorrect jokes... Of course objective measurements are useful for winnowing things like necessarily true statements and necessarily informative statements, but not everything.
I think reddit has a pretty good model. So is Slashdot. They're both obvious improvements from BBS style discussions. But there's a lot more to explore, and this more should be where there is more flexibility. One example where this flexibility is lacking is in funny Slashdot comments. Some of the +5 Funny are great stuff, but I can't just, say, have a Slashdot Jokes section and only look at those (and add jokes to the jokes section). So there's seenonslash.com. But that's a reposting of slashdot comments; in effect, lots of information that could've been more effectively harnessed if the Slashdot system itself was more flexible.
Just my thoughts, but in general I don't like institutions and hierarchies and I'm biased against them. Of course it's insane to abolish all demarcations, but why not allow responsibilities to be expanded or dimished in small increments according to performance? It might take x percent of y to impeach someone in real life, but a virtual system would allow you to, say, have a bit less power if the populace thinks you should have less.
It'd be an interesting experiment if anything.
I wish that were true, but the traditional way is the traditional way for a reason. It worked, providing necessary efficiency in a specific context. Now, I would argue in an 'ideal discussion system' there would be a variety of different contexts that would be modeled in different ways. This means that you have to have flexibility in the individual implementation, which is part of the reason I was exploring the possibility of lisp list strings in transactional databases below on this thread.
The problem with the arbitrarily-tiered system as I see it is that it relies on tagging. This may work in a smallish community but then you still have a problem when what you or I might consider trolls put youtube-style comments into this discussion on social modeling.
The main way that this has been dealt with is to have evolving communities that focus on very technological subject material. By keeping things at a sufficient level of abstraction, one is able to create a new tech-specific language that must be mastered before one can participate. In other words, one has created a de facto tiered system.
The dark side of this is that the need to force discussion into tech-specific language prevents people from having substantial discussion about fundamentals. And fundamentals (like how people discuss things, or would like to discuss things) are often more important than the latest trends in functional programming.
So I would say that Slashdot and Reddit, though good models, encourage a certain faddishness with respect to the abstract language which they enforce.
As for responsibilities being incremental instead of predefined, I have had similar thoughts. I think the problem (as articulated in political philosophical tomes as well as other places) is that the populace may shift unless it has an investment in something in particular. Usually this investment is in property. With the online world one would need something like core users which are in some way invested in the success or failure of a particular online venue. Once one had core users (property owners) then there are a number of fluid mathematical systems one could employ for the evaluation and promotion both of good content and good users.
I was going to post more philosophical musings related what I see as the Turing Project, but I think Pg deleted my last comment on the topic and this post is quite long already.
Digg and reddit do it in the most straightforward manner. Slashdot is good because it actually matches up with some material in conversation theory. The metrics they chose are pretty good. The rating system though, not good enough.
I agree that the tech-specific language acts as a filter itself, but from a different perspective, I realized that Slashdot's modding system's UI and rules are probably a filter too. i.e., if you are willing to learn all the modding rules, and use the modding system, you've passed the "ritual." To the average user, these complications may well be intolerable.
I need to figure out what's tolerable, soon :)
Content rating is done on a color gradient, from white to blue. Blue is good, white is bad, light blue would be okay. Only members can rate content. Rating would be also be weighted according to the karma of the person doing the rating. In other words, members with more karma would give (or take away) more karma per rating.
Each post would receive a default rating based on the karma of the poster, which would display as a color on the white to blue spectrum already described. The color would change as people rate the content.
Not only comments, but also threads could have ratings on a similar basis. This would allow people to set up certain filters so they would view threads with a certain content rating. Probably a person could automatically become a member when they pass some arbitrary karma threshold.
What do you think about this? Are you already working on something 'tolerable'?
One thing though, I am fairly uncertain about the color gradient. In the beginning I used red-green. But some cultures have the order reversed, and total red and total green excite the eyes differently. Then I opted for using colorless-color (white-"gold", since gold is supposed to be culture-neutral), but when I showed some people they were confused and told me to use red-green :/
I also considered the "full functionality after x threshold" but It's opaque to me what would be a reasonable x. I hope I'll be able to make a judgement later, based on real user data.
I'm not sure about 'x threshold' either. If 'karma' assignment is weighted it wouldn't matter much anyways.
As for colors, what about a bar that increases vertically and has multiple segments? Higher is better and darker? This would give people two things to lock onto, just in case color is not enough.
just my two beans...
Really? Isn't that more or less what they did with Pica when they requested that he stop using certain epithets?
I have no idea why this was satisfactory to the reddits, since all it did was add a hilarious dash of ironic political correctness.
There are some very popular forum sites that have existed for years and maintained civility. Any Reddit-type site could learn a lot from them. They almost all have a group of core users who act as moderators and enforce a set of published rules. It's critical that they not make up the rules on the fly, otherwise users won't respect the decisions.
1. The standeard up/down arrow -- does this contribute to the debate, is it a troll etc. 2. The agree/disagree poll. This could be a thumbs up/thumbs down.