Dates covered: 2007-2-20 -- 2023-5-25
Total stories: 178,072 (Should match 5,939 days * 30 stories, seems to come a bit short with 98 fewer stories than expected.)
- States mentioned: 50 (mentions: 1,344)
- Cities mentioned: 109 (checked against list of 330 largest US cities, plus a few additions of my own).
- Sites submitted: 52,687
Top 20 states by raw string-match (this is adjusted below for some overstatements):
1. new york: 316
2. california: 308
3. texas: 98
4. washington: 73
5. colorado: 38
6. florida: 38
7. arizona: 28
8. georgia: 24
9. utah: 24
10. kansas: 23
11. virginia: 23
12. oregon: 22
13. michigan: 19
14. minnesota: 16
15. north carolina: 16
16. ohio: 16
17. alaska: 15
18. indiana: 15
19. hawaii: 14
20. maine: 14
Confounding factors include other terminology, other name matches:
"New York" frequently appears in reference to newspapers or cities:
new york times: 98
new york city: 55
So adjusted NY score: 163
"Washington" is even harder to disambiguate, as it may refer to the state, city (DC), as a topnym for the US / US Government, etc., It's challenging to identify these textually, though the newspaper and monument mentions can be identified:
washington post: 16
washington times: 1
washington monument: 1
Adjusted WA score: 55. And is probably lower.
California is often mentioned by toponym (e.g., "silicon valley", region (bay area), or specific cities (SF, LA, etc.). I'm NOT going to adjust for these but do note the following major occurences:
silicon valley: 399
bay area: 65
california ave: 2
There are a few other more minor examples such as "kansas city" (confounding "kansas"), "oklahoma city", and "iowa city", though those don't affect top results.
Adjusted top-5 state rankings:
1. california: 308
2. new york: 163
3. texas: 98
4. washington: 55
5. colorado: 38
So, answering the original question, Colorado is the fifth-most-loved front-page state.
The 10 least-represented states:
41. new mexico: 6
42. alabama: 5
43. idaho: 5
44. north dakota: 5
45. connecticut: 4
46. nebraska: 4
47. new hampshire: 4
48. rhode island: 3
49. south dakota: 3
50. south carolina: 1
Top-10 US city-name mentions (see confounding factors above):
1. new york: 316
2. san francisco: 192
3. boston: 117
4. seattle: 87
5. hollywood: 76
5. tempe: 69
6. berkeley: 63
7. chicago: 62
8. cambridge: 50
9. detroit: 41
10. los angeles: 30
(This excludes the NYC burough of Manhattan, which gets 36 mentions itself, nudging out LA.)
I've also got a list of most-featured front-page sites, with the ten most frequent being: nytimes.com, techcrunch.com, arstechnica.com, bloomberg.com, wired.com, wsj.com, youtube.com, wikipedia.org, BBC (as both bbc.com and bbc.co.uk), and theguardian.com. The mean votes and comments for each are interesting, with Bloomberg topping by both average vote (255.46) and comments (178.04), from the top-10. (There are obviously sites with much higher mean scores but less frequent appearance, including singletons).
I've broken submitters by freqency, points, and comments as well, with 43,745 submitters accounting for the stories analised. The long-absent nickb tops front-page appearances at 1,322, followed by ingve (1,177) and tosh (679). I've apparently landed 60 front-page submissions averaging 329.00 points and 147.52 comments (both fairly respectable --- I'm apparently not submitting total tosh).
I'm still updating my code and checking data (and found a couple of glitches writing out the above, e.g., whatever I'd done to tabulate city mentions didn't work). Data exist presently in flatfiles, I'm thinking of dumping to sqlite for possibly easier/faster analysis.
A rough and preliminary look at countries represented in front-page titles, again with numerous issues:
1. US: 1350
2. U.S.: 1073
3. China: 634
4. Japan: 526
5. India: 477
6. UK: 288
7. EU: 225
8. Russia: 221
9. Germany: 165
10. Canada: 162
11. Australia: 157
12. Korea: 140
13. France: 116
14. Iran: 91
15. Dutch: 80
16. United States: 75
17. Brazil: 69
18. North Korea: 69
19. Sweden: 68
20. Cuba: 67
21. Egypt: 60
22. Norway: 60
23. USA: 59
24. E.U.: 54
25. Israel: 53
26. Iceland: 52
27. New Zealand: 47
28. Mexico: 45
29. Spain: 43
30. Finland: 39
31. Italy: 38
32. South Korea: 38
33. Switzerland: 38
34. Singapore: 36
35. Mali: 35
36. Chile: 33
37. Turkey: 33
38. Argentina: 27
39. Netherlands: 25
40. Syria: 25
"Cuba" often appears as "Mark Cuban", Jordan is the country most often confounded with a major celebrity and/or IT/tech personality ("Michael Jordan" and "Jordan Hubard" respectively), "Mali" is most likely to be confused with a car or AMD chipset, the United States has multiple possible references (US, U.S., USA, U.S.A., America, American), some of which are confounded (e.g., "North America"). But this provides a rough sense.
Analysis is largely awk, grep, sort, uniq --- standard shell tool stuff. Amazingly powerful though really. Crawl via wget which performed wonderfully as usual. Thanks to HN/YC for not throttling too aggressively.