News.YC statistics - community cohesion
clipboarded.blogspot.com
clipboarded.blogspot.com
http://www.idiagram.com/CP/cpprocess.html
Looking through my own submissions I see that the more intellectual stuff I've posted, while it's done well, has never gotten as many votes as the pithy one-liners.
I suspect you see something that I didn't; how do you approach reading this?
Also, between reading and coming back, you have been downvoted once. I have no idea what is downvote-worthy in your post... sometimes I just don't know what I am missing -- or if there is anything to miss in the first place.
No, seriously, all the reasons you present a perfectly legitimate reasons for up/downvoting and are reasons that I have considered at various times.
You can never downvote article submissions; they can only be upvoted. Downvoting sumbmissions has merit in some cases (removing spam), but in practice it has been used on other sites to kill everyone's submissions except your own, rapidly leading to a downmodding war where the winner is whoever is more committed to their karma score, instead of the one who is submitting the best articles.
Normally people solve problems using some variant of the rational decision making problem. That is, define the problem, define assessment criteria, brainstorm solutions, choose solution, implement solution, measure solution against assessment criteria.
A wicked problem, on the other hand, is a problem that can't be clearly defined and has no clear answer. For example, what is the best way to design a car? If you maximize mpg then the car isn't as safe. But if you maximize safety then the car is heavier so the mpg isn't as good. And both safety and mpg affect style.
The idea of the idiagram thing is that while the rational decision making model may be the best way to solve well-bounded problems, it isn't a very good way to solve wicked problems.
This is roughly where the idiagram page leaves off, but it footnotes the book Wicked Problems and Social Complexity, the first chapter of which is available online. This explains dialog mapping, which is a method of discussing wicked problems in group meetings using mindmaps.
Why is this important?
If you've been to a lot of discussion forums on the web, you probably realize that forums with nested comments tend to have more rational discussion than forums with flat comments. After all, it's just easier to reply to specific points of an argument with nested comments than it is to fisk one with flat comments. You can go back and forth on one point, and then go back up in the discussion later and talk about a completely different point raised earlier.
Trying to solve wicked problems in real life meetings without dialog mapping is sort of the equivalent of an flat discussion on the web. You are sitting around and someone comes up with a complicated proposal. The next person responds to whatever they see in it. And then the other points in the original comment become more or less buried. However if you mindmap all of the points of a person's dialog, then people are able to use the visualization to jump back and forth during the discussion and gain perspective on what's important and what's less important. This ultimately means better solutions to wicked problems in less time, with more focus on the problem and less focus on the personalities involved.
The idiagram then is a way of visualization the final output of the brainstorming via dialog mapping process so that all involved can be on the same page when in comes to understanding the solution and each person's responsibilities and how success will be measured. (The whole submission is kind of recursive, because it's a picture about the four steps of the WP solving process, and making a picture is step three of four.)
Anyway, that's what I see in the submission. If you're the kind of person who sits are and thinks about how you can have brainstorming sessions in real life with the same abstract ideas and rational thinking that often arise online (because of the asynchronousness and nestedness of the internet), then this is kind of a cool thing.
I don't know to what extent a difference it makes, because everything I see, I think is obvious (forgot the term for this): I think holistic and heuristical thinking is the de facto, and most obvious way of dealing with multidisciplinary problems; in this case, I believe this falls into the same category as the "wicked" problems you describe. It seems like everybody on this forum has a fair amount of exposure to psychology, such that many concepts that I learned in a formal setting are in fact common sense. I have no idea how true this is, but I try to avoid stating the obvious, so I generally keep quiet about it.
I still suspect the following would be nothing new to you, but it's a different route of thinking so I'll write anyway.
A problem that "cannot be clearly defined" and has "no clear answer" is generally solved heuristically. This is an area where hard-earned experience, and not necessarily genius, pays off. Warfare is a good example, because it involves not only logistics but also understanding the enemy's culture and psychology and estimating their intelligence. Heuristics come from efficient processing of declarative and episodic memory, which are, very obviously, the result of accumulated experiences across diverse fields.
So regarding how to best design a car, it is logical to think that an engineer would be better than an artist, because they know what is required to make a car work, but an engineer with an art background is better than one without, because they can efficiently (i.e. long-term memory retrieval) co-synthesize knowledge pertaining to design and construction. This must also be the reason why pg says "hackers who can design are dangerous."
Your points about forum dynamics follow the same ideas. A flat discussion, which interferes with thought organization, forces readers to synthesize information as a whole, which places a larger stress in keeping relevant information in memory. A threaded discussion, on the other hand, has less stress on memory, because certain steps can be stored as logical patterns that draw from existing memory.
A very simple thought experiment to illustrate this could be as follows. Imagine a threaded discussion with these messages: man. suitcase. taxi. airport. airplane. country. building. meeting.
Next, a flat discussion: man. window. smoke. lake. chocolate. tree. bulldozer.
Now paint a mental image out of the information from these two threads. The first follows a very logical progression, is easier to remember, and can be painted in a fairly predictable manner. The second one seems more disparate, requires more memorization, and the end result is harder to imagine, a la the "wicked problem."
Hope I'm not too far down a tangent... I guess the relevance to the diagram is, it doesn't matter what diagram or map you use; the goal is simply to get all the facts into everybody's heads simultaneously. A mind map could work. A list would work. A powerpoint. Or, based on the type of stimulus, a mixture of things (my interest).
Where does lateral thinking come into this process, and has it been useful in your experience? I tend to think counter to the conventional wisdom, and people tend to think such ideas weird.
While it's normal to say this is a good thing, I'm not sure it is and am curious if there is any data on the usefulness of lateral thinking.
Let's say you post an amazingly funny technology joke and picture each day, and a friend posts an amazing, in-depth discussion of something that most of the readers would want to consume.
Who do you think is going to score all the points? Certainly not your friend with the in-depth articles.
So does that mean the the community is more represented by people who post quick, funny barbs? Are we a group of late-night technology comedians?
I understand the joke analogy is at the extreme, but you can make the same observation comparing, say, 3-paragraph gossipy blog entries with 50-paragraph in-depth articles. Same principle applies.
Since you mention that it actually provides insights after prolonged meditation, I wonder if it would be acceptable as a Mandala, though.
I think if Marshall Clemens expressed those 5 web pages as 5 paragraphs, you would see that he's either not really saying much or is saying something that has plenty of counterexamples that aren't apparent in the pretty picture.
For example, it's kind of ironic that I find none of the diagrams live up to the claims he makes about his Visual Models™ here http://www.idiagram.com/CP/visualmodels.html
It's nice to have something that describes that whole process -- faced with a toughy problem? This is what to expect, and lest you forget, here's how you can approach it.
I think part of the reason that news.yc is interesting is that there are a number of non-overlapping interest areas within a tight niche. Think about that. Diversity within a narrow field. I like that.
I'd like to think that the factors you mentioned (traffic, time of the day, controversy of the post) iron out statistically (since all posts are affected by those, there is an argument to ignore their respective influence).
This initiative makes me wish it is only the beginning: I want some more stats! For example:
- It would be interesting to see what's the points per post (PPP) of the leaders, compared to the PPP for most of the others (I agree with you that in the dataset, the leader to user ratio is too high). Maybe one could have a plotted distribution of PPP's across all users.
- I know HN has 8000 uniques/day, what is the proportion of those that are posting? e.g. how many of those users with less than 5 posts?
- also interesting is the proportion of points between posts and comments, per user. Different profiles: some people discuss a lot, some people post a lot, as it was often repeated in discussions. But what's the proportion of those 2 groups?
Or what about looking at voting habits? What's the average/min/max/etc up/down voting ratio?
If you showed to what degree showing more statistics, particularly about new postings or first-page postings would change behavior?
You've certainly embodied the philosophy of Edward Tufte:
http://www.amazon.com/s/ref=nb_ss_gw/102-8193666-1218533?url...
There must be many cofactors that are hard to capture in an analysis like this. There's the "binaryness" of making it to the front page. There's also "time of day". A very interesting submission posted at a time when those who would be interested are not online may never gain enough traction to make it to the first page. A time-of-day analysis would certainly be interesting.
I feel like starting my own social news website just to have a play with different algorithms!
Neither, I'm sure.
(Funny. lately when I get sarcastic here, people think I'm serious, and when I'm serious, people think I'm joking. I guess I need a little more practice writing to strangers.)
Scatter diagrams are one of the greatest underrated ways of conveying information in a heartbeat. You've done a great job of it.
You've also opened up a can of worms. There are so many ways to look at this data, you can keep yourself busy for quite a while. I suspect that won't be a problem, considering how much you already care about this. Keep up the good work and keep us posted.
The real problem is what most of us would call a fall down today, that will be called tommorow (and interpreted as) a rise up from other people (thus the extreme difficulty to create efficient moderating systems based on quantitative data). You know your community is irreversibly "corrupted" when this kind of people becomes the majority, and this usually happen surprisingly quickly with the faster growth of visitors.
For some reason I have some hope for HN because of PG being behind it, he kinda knows what he wants and certainly knows what he doesn't want. Still, this might be one of the thoughest challenge he'll be faced with if HN keeps growing. I've known no community that escaped this syndrom yet.
I'd be interested in seeing similar analysis on a per-comment basis.