Hacker News Reader

Comment by mxfh | Hacker News Reader

There's a xkcd for that: http://xkcd.com/1138/

"Pet Peeve #208: Geographic profile maps which are basically just population maps."

> To protect privacy, actual tweet locations have been aggregated to the county level and normalized by number of tweets.

So, given that this is a rate-based map, the XKCD doesn't apply

although it's not clear in the language used, the data seems to have been normalized per Tweet:

"Orange County, California has the highest absolute number of tweets mentioning many of the slurs, but because of its significant overall Twitter activity, such hateful tweets are less prominent and therefore do not appear as prominently on our map."

Except that this isn't, which is what makes it interesting.

I'm not sure it's interesting. It reflects something so complicated that it's hard to learn anything from. If you interpreted it as a heat map of racism, you'd conclude that the least racist region of the country is the mountain west outside of Idaho. Utah and Montana are almost spotless. It might be closer to interpret it as a heat map of "racist passion, overtly expressed" but even so I bet a lot of it has to do with the demographics of Twitter adoption. I guess sitting here guessing is kind of interesting, but I feel like I'm projecting my own assumptions onto it to try to figure out what it means rather than learning anything from it.

So this explains why LA disappeared since the counties there actually have some twitter active people living in it to help normalize it. When zoomed in, in the east there seems to be some halo effect at work were rural counties around bigger metros (Dallas, Atlanta, North Eastern Seaboard), with a lesser volume of tweets (low sample size) appear more active.

When zoomed out the 100th meridian population divide is as obvious as it could be. So At the global level normalization is pretty much not in effect. This might likely be caused by the not uniform distribution and size of counties in the US.

A better approach might be binning in equal area regions, with a threshold for low sample sizes.

None of the major population centres are well-represented in that map.

Dallas basically is. Also weird to see Austin being worse than Houston.

I wonder if that's a flaw in the model by not normalizing over how popular twitter is in that area. Seems like it's more useful to know what percentage of tweets are hateful instead of counting sheer number.

ie: maybe there's just way more tweeting in Austin.

[edit: I'm apparently wrong. @ronaldx points out below a statement that suggests this is already happening and is why Orange County doesn't shine brighter]

Come again? It looks to me like the map tags almost every one of the major population centers. Southern California seems abnormally low, but that's just about it.

Chicago?

true in many instances, but i'm not sure it applies in this case.

This is actually the opposite in some states. My post below, for example:

The three hottest counties in New Mexico and Idaho have populations under 34000 combined. One of them is just 2900 people.

A low population makes it easy for a small number of users to dominate the statistics.

Yes and a comparison of similar low population counties on opposite sides of the Mississippi river shows staggering difference in claimed result yet no real difference in culture or demographics. We're being trolled, gentlemen. It only remains to figure out why, for what purpose. That troll study might end up being nearly as interesting as the supposed data would be, were it real.

I don't think we have enough information to draw any conclusions. In many cases, a single highly active user could be pushing a whole county (which also means that it isn't very suspicious to find a county without one).

The number of users tweeting hate and how connected they are to others is probably a more interesting map.

"I don't think we have enough information to draw any conclusions."

I would disgree in that we have a geographic claim that, for example, Nebraska and Iowa supposedly have spectacular difference in hate levels, like order of magnitude, yet simultaneously claims that Salt Lake City and San Fran are equally gay-friendly. And on a micro level, the hyper-segregated highly urban area to my east has no racist tweets at all, yet farmville east of the mississippi is supposedly a hotbed of racism but magically it all disappears west of the mississippi. I've lived in both Wisconsin and Alabama and anyone is seriously claiming the racial climate is basically identical? LOL.

Nope, we're being trolled here. I do agree with you that more data, or more analysis, would make it more interesting.

The map is difficult to interpret. You are making the mistake of conflating the national average rate with "no racist tweets at all". It is a jump to say that the difficulty interpreting the map results from intent of the creators, which is the line I would draw for trolling.

It could be titled something like "The Apparent Relative Visibility of Hate in Tweets at the County Level". "The Geography of Hate" isn't such a far jump from there, especially if you are just trying to show that the tweets are coming from all over.

Reply on news.ycombinator.com