Hacker News Reader

Comment by eanzenberg | Hacker News Reader

CLT is abused among data scientists quite often, and usually due to the points in the article.

- Convergence takes infinite time when the sampling variable has infinite variance (imagine a random number generator).

- While the distribution of an aggregate statistic tends to a Gaussian, most underlying distributions are not and most stats is done using the underlying raw data.

Junior data scientists assume everything is Gaussian (sometimes attributing it to CLT) when doing linear regression when most distributions being modeled are not. Then, analysis done using variance on the coefficients is meaningless because the assumptions are incorrect.

View on HN

Everyone assumes things that aren't true, not just "junior" data scientists. There is not standard methodology for non gaussian non independent random variables.

Linear regression doesn't require or assume the input data values to be normally distributed, just that the residuals are normally distributed and have constant variance.

Linear regression doesn't require much, honestly, just non-null numeric data. My point was more on the interpretations of the outputs.

The constant variance part is important though. If you apply linear regression to a severely heteroscedastic dataset, the inferences you obtain from the model will be incorrect.

Further, those assumptions are only necessary for confidence intervals and hypothesis tests.

This comment is surprising to me. Most data scientists use results and measures of statistical significance provided by the program they're using, which accounts for the distribution used. Do you have examples where either data scientists are not presenting aggregate statistics or where someone is using the wrong kinds of p-values?

To your specific examples - the coefficient of a linear regression is distributed normally (?). Similarly, we know the expected distribution of most maximum likelihood estimators (logistic regression, etc.), and programs will give you the right p-value.

Of course, omitted variable bias is still a problem and it is possible to mis-specify your model. However, I think most data scientists are presenting aggregate statistics (means, regression coefficients) like you said, and that we have a pretty good handle on the underlying distributions.

>which accounts for the distribution used

what does this mean? are you trying to say the code uses the empirical distribution as a proxy for the true distribution?

Mispecification error is hard to deal with.

For example, there were times where features were excluded because their "p-value was too large to be significant", regardless that the underlying distribution was not Gaussian. p-value from a t-test, like in lots of regression software, requires Gaussian distributions.

If the data set is large, then the coefficient is approximately Gaussian, because of the CLT. This is one appropriate setting to use the CLT (unless you think you are in a setting where the CLT doesn't hold, such as infinite variance).

For example, lets say height is a feature in your model. No matter how big the size of data, it will never be Gaussian, it is bi-modal. So the t-test in regression won’t be valid. Most ml is done on raw data.

If the t test is on a regression coefficient, then the sampling distribution is approximately Gaussian (for big enough data). It doesn't matter how many modes that the original data feature has. This is standard asymptotics in hypothesis testing.

No, a t-test makes no assumptions that the underlying data is Gaussian. Again, most ml is done on raw data. If the raw feature is bimodal then the raw data is bimodal.

I can't figure out if you are agreeing or disagreeing with me. If you do non-penalized regression on raw data, then the t statstic will be approximately Gaussian, even if the raw data is bimodal. This follows from the CLT.

What is the standard deviation of a bimodal distribution with modes at 0 and inf? Is that a meaningful stat?

> Convergence takes infinite time when the sampling variable has infinite variance

When does that really even happen in practice though?

> most distributions being modeled are not

That a rather assertive statement. I dare say most distributions are (due to CLT) Gaussian.

Distributions are only assumed. There is no such thing as truly normal data or sampling distribution.

Sure, though that's a bit like saying there aren't any perfect squares only things you assume are squares.

There is a difference. Distributions are tested in practice, for example, by using a goodness-of-fit test. Often if the hypothesis of normality is not rejected, people think this implies normality.

> That a rather assertive statement. I dare say most distributions are (due to CLT) Gaussian.

I have been doing stats/ML/data-science for more than a decade. The above has rarely been true ever in the datasets that I have looked at. Unless of course the raw data has been transformed in ways to make them look Gaussian. Gaussian is the exception, not the rule.

Reply on news.ycombinator.com