Hacker News Reader

Comment by dkersten | Hacker News Reader

I guess you ignored their disclaimer beside the editor statistics?

From their methodology section:

> The data include responses only from the official Python Software Foundation channels. After filtering out duplicate and non-reliable responses, the data-set includes more than 18,000 responses collected in October and November of 2018 via promoting the survey on python.org, the PSF blog, the PSF’s Twitter and LinkedIn accounts, official Python mailing lists, and Python-related subreddits. No product-, service-, or vendor-related channels were used, in order to prevent the survey from being slanted in favor of any specific tool or technology.

View on HN

If I remember correctly, I believe the JetBrains / IntelliJ / PyCharm brands of the sponsor were very visible when taking the survey.

That might have influenced significantly in the selection of the Python users taking the survey: why answer a survey setup and sponsored by a private company you are not really familiar with and that you don't necessarily trust as a result?

There is also a second selection bias: the people who follow the PSF and the official python.org communication channels are probably nerdier than the average Python programmer. This is reflected in the OS statistics. The survey results suggests than 53% of the Python users do not use Windows at all.

On the scikit-learn online documentation, after removing the mobile traffic we have: Windows at 61%, macOS at 23% and Linux at 15%.

On stackoverflow, the primary OS statistics are: Windows at 47%, macOS at 27% and Linux at 23%.

oh come on. 18000 responses.

This is the js survey results all over again: no. Unless you can statistically prove the results are biased, you don’t get to ignore the results because you dont like them.

Finding data points with no methodology that contract the survey result does not invalidate the survey results.

Thats. not. how statistics work.

A great deal of effort was put into this survey, and the stats you’re looking at are more likely biased than the ones in this survey.

The stats and the methodology here are clearly documented; if you want to argue with them, be specific and provide concrete statistical proof for your assertions.

Specifically, why do the stats you have prove anything, and what confidence do you have that they are representative?

If you're that strict then this survey is also useless as you cannot prove anything about general population of Python developers.

I think the point here is people tend to reject survey data because they can only see some tiny minimal subset of the data and it doesn’t match in aggregate.

In this situation often the smaller dataset is wrong.

...not always. But often.

Human intuition based on limited data can seem compelling, but it’s always worth acknowledging you might be the outlier.

18000 respondents is a lot, especially when a specific effort has been made to sample from various sources.

The parent post didn’t even bother to check their own biases.

Another thing that I've noticed is that often what we, humans, feel makes intuitive sense (and therefore must be correct), when you actually look at the data, is often very wrong. Just because it makes sense to us doesn't mean its correct. Basically, humans are naturally biased not only towards certain held beliefs, but also towards things that just "sound" correct, regardless of what the reality is. It makes sense though, if an explanation of something is understandable to us, then we can think about it logically (or otherwise) and come to conclusions, but unfortunately often things that are understandable are also based on flawed assumptions.

I don't know if that's the case here, but I do often think that people gravitate towards the first thing that they can understand that seems to, at a glance, check out, without investigating.

Your statistics don't necessarily contradict this survey. Scikit-learn users are a subset of all Python users. Maybe data scientists are more likely to be using Windows than other Python developers? Its impossible to know from either set of stats.

Similarly, stackoverflow has a lot more users than just Python users, so it says little about how many Python programmers use Windows.

Reply on news.ycombinator.com