Hacker News Reader

Comment by jononor | Hacker News Reader

Gah, those wordclouds are horrible attempts of analyzing "what the questions are about". There is no attempt at normalizing term frequency, or removing non-informational words. So words like "get" and "using" get a high score.

It may also seem that the text analyzed is the entire post including metadata. For instance "closed" and "duplicate" both show, maybe just be because of "closed as duplicate"... Maybe even including the answers. Which could explain why "pip" would show up so often.

The summaries are also pretty off. "vector" mostly because C++ is about 3D graphics vectors? No, probably just a whole lot of std::vector...

View on HN

It's much better to just use a table of phrases/keywords, even if it takes a little more effort. Phrases are specific and have clear intent - using only the most common words leaves out critical context!

Some of the simple words and their prominence can be amusingly informative, though.

Take data structures for example. "string", "array", and "object" are about as equally prominent in both JavaScript and Ruby (where the dictionary is called "hash"). In Python, however, "string" and "list" far outweigh "dictionary" and "object", which probably says something about what kind of data structures Python developers deal with the most in their lives. Meanwhile, C# and Java seem to be all about strings -- Are people just casting everything to string because they don't want to deal with strict types? -- and PHP is the only language where more people feel like they need to ask about arrays than they do about strings. Which is not surprising since PHP uses arrays for basically everything.

> In Python, however, "string" and "list" far outweigh "dictionary" and "object", which probably says something about what kind of data structures Python developers deal with the most in their lives.

This statement is not true at all. There are plenty of developers that don't post questions to or answer questions on StackOverlow.

Not saying anything about the parent comment, but the method of studying a large population by using a sample size is pretty common and time tested.

Yes, but SO participation is self selecting (self-selection bias), and is not a random sample of the larger union of non-SO and SO population.

And the 'statistics' on the data was presented as a word cloud, which is not a time-tested way to understand statistical relationships.

> but the method of studying a large population by using a sample size is pretty common and time tested.

Yep, and so is abusing it and making the wrong conclusions. It's equally as likely that dictionaries require far fewer questions than strings either because their methods are more intuitive or the questions most people come up with have easily searchable existing answers.

Are people just casting everything to string because they don't want to deal with strict types?

No? How would that even work? It's not really a sensible way to draw conclusions about what people do with the language. For instance, you're much more likely to type 'Object' in Javascript than you are in Python.

>”string” and “list” far outweigh “dictionary” and “object”

“Dictionary” probably loses a ton of hits by the split with “dict”. And “object” is basically a dead word in python now.

Plus, string and lists are typically hit by newcomers before they are taught dictionaries and classes. A lot of SO questions come from homework.

Reply on news.ycombinator.com