Hacker News Reader

Comment by bunderbunder | Hacker News Reader

It's unparalleled in that there is very little that's even similar out there.

I'm less convinced of its actual value. The Twitter-based papers I've read seem to all have two things in common: First, they all use Twitter data. Second, they all seem to desperately want to present observations about how people interact on Twitter as being universal, even though nobody seems to be able to come up with a theoretical or empirical reason why that should be the case that's more compelling than, "We we have to pretend it is in order to get published."

View on HN

Yes. It also bothers me that quality news sources, like NPR, have fallen into the trap of presenting Twitter as representative of the general population.

They often report about how some group of people on Twitter are angry about something -- as if that matters. Reddit has almost as many monthly active users lately, yet no journalist makes the mistake of reporting on Reddit comments as if it's generalizable to the larger population.

Most of the news reporting I've seen cast on individual Reddit communities. I don't think such a thing can be done as easily as twitter except through hashtags, thus the coverage on twitter being more twitter-wide hashtagging rather than sub-tweet communities.

What's interesting is that, even though they're not well-defined, there certainly are specific communities on Twitter. There's tech, and sports, and science, psychology, politics, music, etc, and each of those can be further divided. I'm sure that if Twitter put in the work, they could suss out all these communities based on who is following who and what they tweet about. (In fact, I'd be surprised if they aren't already doing that.) They could then make these communities accessible to the public.

Twitter currently has lists, but the problem is that these lists aren't crowdsourced (to my knowledge). They're maintained by individuals, which is unsustainable, because one person can only do so much work keeping a list accurate and up-to-date.

I'd be pretty sure that "Who to Follow" is based on some sort of social graph though probably in a simplistic way, e.g. lots of people you follow follow this person and you don't.

Yeah, that's the only thing I've seen from them in this domain. It's just woefully lacking imo. I don't want to pick people to follow individually... I want to pick entire communities of people to follow.

So essentially lists created using social graph algorithms. I probably wouldn't use it personally but I can see how it could be useful.

> I probably wouldn't use it personally but I can see how it could be useful

This describes virtually every announcement I've heard from twitter.

Twitter is a cheap way to get response quotes for an article without asking anyone any questions. It's gotten so ridiculous that some news "content" is just a series of embedded Tweets. Serious news organizations should band together and restrain themselves from this silly practice. It violates one of the most important rules of interviewing, which is to know who the source is and what her biases might be so that the quote can be presented appropriately.

There's a lot of lazy journalism looking for a quote, any quote, because they're required to have one in a story they're writing.

This isn't new. When I was an analyst, some reporters would call me for a quote about something. There were times when I didn't have anything to say because I didn't know anything about said thing. But some would pester me for a quote anyway so they didn't have to give someone else a call.

Twitter does seem to have become a particular crutch though.

The BBC was one of the most egregious pioneers in soliciting meaningless viewer responses in the pre-social media era. They would go so far as to kill 30+ seconds of airtime to soliciting/demanding "participatory" viewer responses that were invariably inane. On the plus side, at least it led to some good satire [1].

[1] https://www.youtube.com/watch?v=OQnd5ilKx2Y

Auntie Beeb's insistence on such complete drivel has killed all the joy and interest. I find it utterly unlistenable much the time.

yet no journalist makes the mistake of reporting on Reddit comments as if it's generalizable to the larger population.

You should stop by /r/austin sometime, and just lurk. The amount of local reporters looking for some kind of pulse to put their finger on and even those nu-'journalist' sites (Vice,Vox,insert here) obviously mining for quotebait is absurd, and the community has been calling it out.

Of course, /r/austin isn't a remarkably interesting place to be anyway for a lot of reasons...it's just interesting to watch the "Austin" phenomenon manifested there. Different story though.

  quality news sources, like NPR

NPR is CNN marketed towards psuedo-intellectuals by replacing common terms with sophisticated sounding vocabulary. Most of the pieces that discuss topics I am heavily familiar with are shallow and often misrepresented.

I say this as someone who really wants to like NPR and struggles to find any unbiased news source.

I'm sympathetic to claims of NPR bias (still a big fan), but rather than seeking an "unbiased" source, I want a thoughtful, reasonable, open-minded and well-mannered conservative news source. It does not exist.

You might look into the magazine The American Conservative. It's far from perfect (Pat Buchanan uses it as his soapbox from time to time, for starters), but on the whole it's pretty readable.

I heard one of the "big data social scientists" to claim that they can now replace the population with the sample (as this is what the Twitter really is) because of, you know, big data.

If you have a demographic model of twitter users and appropriately re-weight, why is this a bad approach to take?

If I understood correctly then this was considered as a not necessary step.

Now given the Twitters nature then is there such a model that is even approximately accurate?

Or in other words, what relevant data Twitters collects and how accurate it is?

Because you don't (and probably can't) know that this model is accurate? Also, there are a lot of populations which are not present on Twitter at all, and more which are not present in meaningful numbers.

Such models presumably can be tested in some sense. E.g. build some sort of predictive model of elections in each state using Twitter users as a surrogate, see how well you do.

I'm not claiming that such a model is guaranteed to be perfect or even as good as a pollster, but it sounds in principle like a reasonable thing to do, if you take appropriate precautions to reduce bias.

At least one can try definitely.

There's thousands of sites serving the same or similar purpose. Twitter is only unique because it has a lot of users.

It's not really about whether or not the site works similarly to Twitter. It's about the neat stuff people can build with something like Twitter data. I don't think you could do something like http://blog.idl.ssrc.msstate.edu/?p=42 with Plurk data and get anywhere near the same kind of attention for it.

For that kind of thing, having lots of users is the key feature of interest.

Is attention the goal? I've been struck by the number of articles and research projects that are some version of "we got lots of tweets and graphed them" and have totally failed to go the "so what" step.

Why should we care about this project? If you want to use this to find power outages using Twitter, why not just do that directly with e.g. grep (also, more importantly, are these outages about which the utility company doesn't already know?). Having big data (related: 12 Million lines of text isn't big) isn't cool in and of itself. Doing things which help with that data would be cool.

Similar like in Sublevel?

Reply on news.ycombinator.com