As I've commented on HN over the past month or so and shared with dang via email, I've been running some analyses on the HN front page based on a full history[1]. One aspect I've looked at is the "flamewar detector", or as dang's described it elsewhere, and "overheated discussion" metric[3] --- not
necessarily bad, but a sign that things might be getting tendentious. I've taken to calling it "spiciness".
In particular, it seems to me that this might penalise stories in which there is a genuine level of disagreement or contentiousness, or which are socially, politically, and/or culturally contested.[4]
As I understand, it kicks in where a post has >= 40 comments and comments > votes.[2]
Given that I have ~188k front page stories with vote and comment counts, it's pretty straightforward to calculate the comments:votes ratio, and look at patterns that emerge.
Looking at means, we can get, first, the overall ratio and its percentile distribution, and the overall story position (not surprisingly, 14, which is close to 15, the midpoint of the 1--30 range):
Overall ratio
n: 115908, sum: 73618, min: 0.25, max: 21.00, mean: 0.635141, median: 0.52, sd: 0.419485
%-ile:
5: 0.28, 10: 0.3, 15: 0.33,
20: 0.35, 25: 0.38, 30: 0.4,
35: 0.43, 40: 0.46, 45: 0.49,
55: 0.56, 60: 0.6, 65: 0.64,
70: 0.69, 75: 0.75, 80: 0.83,
85: 0.92, 90: 1.07, 95: 1.34
Note that both the mean and median ratios are well below the detector threshold of 1.0. By standard deviation (percentage of values included with a normal distribution in parentheses):
1σ: 1.055 (68%)
2σ: 1.474 (95%)
3σ: 1.894 (99.73%)
Note that the normal distribution values seem to agree with the calculated percentiles. And that HN's flamewar detector kicks off for stories roughly 1σ > mean spiciness.
I pulled up the list of the 40 most "ratioed" (or "spiciest") HN threads of all time:
rank ratio date-pos year pos DoW user vote com site
1 21.00 2007-4-18.18 2007 18 Wed juwo 5 105 juwo.com
2 10.83 2008-2-4.26 2008 26 Mon eusman 6 65 n/a
3 9.33 2007-4-26.26 2007 26 Thu juwo 3 28 juwo.com
4 9.14 2007-7-31.13 2007 13 Tue rokhayakebe 7 64 n/a
5 9.00 2007-7-22.21 2007 21 Sun gibsonf1 3 27 streamfocus.com
6 7.50 2007-4-20.25 2007 25 Fri jkush 6 45 n/a
7 7.50 2007-6-1.10 2007 10 Fri keiretsu 6 45 n/a
8 7.33 2007-3-26.20 2007 20 Mon BitGeek 3 22 n/a
9 7.29 2008-1-27.13 2008 13 Sun wumi 14 102 n/a
10 7.27 2008-7-22.29 2008 29 Tue BenS 15 109 n/a
11 7.22 2008-9-25.17 2008 17 Thu DanielBMarkham 23 166 n/a
12 6.79 2008-11-14.19 2008 19 Fri bbuderi 24 163 xconomy.com
13 6.77 2008-3-28.17 2008 17 Fri btw0 13 88 n/a
14 6.57 2007-10-9.20 2007 20 Tue _h4xr 7 46 f2s.com
15 6.50 2007-10-13.15 2007 15 Sat robmnl 6 39 n/a
16 6.50 2007-11-3.16 2007 16 Sat breck 4 26 qcrunch.com
17 6.50 2008-1-18.13 2008 13 Fri german 14 91 n/a
18 6.33 2007-12-10.24 2007 24 Mon kf 6 38 independent.ie
19 6.29 2007-9-5.10 2007 10 Wed rokhayakebe 14 88 n/a
20 6.25 2007-7-1.13 2007 13 Sun danw 4 25 n/a
21 6.20 2007-5-24.15 2007 15 Thu bootload 5 31 n/a
22 6.20 2007-6-11.9 2007 9 Mon danw 5 31 n/a
23 6.07 2007-2-21.1 2007 1 Wed pg 262 1590 n/a
24 6.00 2007-11-13.30 2007 30 Tue trekker7 5 30 n/a
25 5.99 2022-5-26.8 2022 8 Thu LoveGracePeace 280 1678 nih.gov
26 5.91 2008-5-5.30 2008 30 Mon rrival 11 65 n/a
27 5.85 2008-4-1.15 2008 15 Tue robmnl 27 158 n/a
28 5.82 2007-8-19.11 2007 11 Sun kf 11 64 n/a
29 5.80 2007-8-21.4 2007 4 Tue zenobo 20 116 n/a
30 5.78 2007-4-6.1 2007 1 Fri kkim 77 445 paulgraham.com
31 5.77 2008-4-18.10 2008 10 Fri Flemlord 26 150 n/a
32 5.75 2007-11-18.18 2007 18 Sun rokhayakebe 8 46 n/a
33 5.70 2008-2-12.22 2008 22 Tue theremora 10 57 n/a
34 5.64 2007-10-17.9 2007 9 Wed altay 14 79 n/a
35 5.55 2008-1-17.18 2008 18 Thu sspencer 11 61 n/a
36 5.50 2007-10-23.25 2007 25 Tue DanielBMarkham 6 33 whattofix.com
37 5.50 2007-4-18.25 2007 25 Wed yaacovtp 4 22 n/a
38 5.50 2007-4-7.17 2007 17 Sat me_jobs_r_u_woz 4 22 n/a
39 5.42 2008-3-4.29 2008 29 Tue kulkarnic 12 65 n/a
40 5.42 2008-5-24.7 2008 7 Sat _bbks 38 206 n/a
"n/a" site is a post without a URL, typically an "ask" or "tell" thread.
Eyeballing that list, one pattern that is suggested is that high-ratio posts seem to have relatively high story position values (that is, they appear low on the HN front page), notable exceptions being #23 (pg's 2007-2-21 request), and #30 (kf's 2007-8-21 post, a pg essay "Microsoft is Dead"). I'd done further analysis of story position by spiciness and found that in general for stories which make the front page, higher spiciness actually correlates slightly with a higher page position (lower StoryPos value). I suspect this is due to specific moderator interaction, that is, those stories which survive do so because they're deemed appropriate to HN.
Another is that high-ratio posts are notably clustered in 2007-2008. Frequency analysis suggests that flamewar detection was implemented around January, 2009 as spicy front-page posts thin markedly after that month. The one item appearing after that date ... ticks numerous hot buttons (2022-5-26.8).
Because of how I'm collecting and analysing data I don't have HN or article URLs directly available, though it's possible to determine the article from the front-page archive. "2008-5-24.7" corresponds to the 7th-position story on the 2008-5-24 front page archive: <https://news.ycombinator.com/front?day=2008-05-24>
I also took a look at the highest and lowest ratioed sites, with >= 20 front-page appearances, appearing in a follow-up comment.
Overall, the detector seems reasonably justified, though not infallible. It's a much better indicator than I'd thought it would be.
________________________________
Notes:
1. 20 April 2007 -- 21 June 2023 presently. I re-up the archive periodically, but haven't fully automated this. Results change little incrementally.
2. Via ColinWright 2 years ago here: <https://news.ycombinator.com/item?id=25871251>
3. <https://news.ycombinator.com/item?id=16020089>
4. As it happens, HN tends to downrate each of these topics as dang's discussed over the years. Links ... not immediately available.