Hacker News Reader

Comment by yummyfajitas | Hacker News Reader

This isn't a book about supervised learning, from what I can tell. Based on my reading of the murder mystery and the skill assessment, it's about defining models based on your understanding of the underlying system and then fitting them to the data.

This is a lot closer to classical statistics than machine learning.

View on HN

rather than downvoting i'm actually curious why you think unsupervised learning is not ML?

there'd be so much less noise in these comments/discussions if we just did away with vague and illdefined labels such as ML or AI

To me at least, the major distinction between "classical statistics" and "machine learning" is that machine learning" strives to work independently of the underlying distribution while classical statistics tries to model it.

I.e., a statistician doing linear regression assumes that reality is linear (or at least differentiable) in the region of interest. A convergence proof of linear regression will use this assumption.

A machine learning practitioner does NOT assume reality actually has a random forest out there in the world somewhere, and as a result needs to prove far more general (and less accurate) convergence results for the random forest.

From what I can tell, this book falls into the former category.

> assumes that reality is linear

The assumption is that a particular relationship is reasonable to model as if it were linear. No one believes reality is strictly linear.

I've read your posts enough to believe you know how linear regression works. I'm criticizing your comment because it encourages a misunderstanding of traditional statistics as having nonsensical assumptions.

Out of curiosity, was my caveat "(or at least differentiable) in the region of interest" insufficient for that purpose?

I certainly didn't mean to imply that statistics has unreasonable assumptions. Merely that it tends to have stronger assumptions - and more accurate results - than machine learning. Personally I'm a huge fan of classical statistics and think it's currently underappreciated.

The caveat doesn't work for a technical reason and a more important practical reason. Most relationships, even ones that aren't proper functions, can be transformed into a linear model. An absolute value function is non-differentiable for one value of the input, but it'd be perfectly fine to model with linear regression. More importantly, the audience I worry about isn't the type to pay attention to parenthetical notes using jargon. Linear is somewhat accessible jargon, but differentiable is less so. I'm not claiming that I write clearly, but I aim to write such that I don't need caveats.

Yes it really is underappreciated. As quoted by other comments, "Most businesses think they need advanced ML and really what they need is linear regression and cleaned up data". A significant portion of businesses currently investing millions in ML should basically hire a couple of statisticians and get over it.

To be fair, the fully loaded cost of a couple statisticians (ones who can code, or combined with an engineer assistant) might be half a million or more annually.

>A machine learning practitioner does NOT assume reality actually has a random forest out there in the world somewhere, and as a result needs to prove far more general (and less accurate) convergence results for the random forest.

Of course, most of the time nowadays, "throw a neural network or an SVM at it" doesn't really require strong convergence results... even though there are some nice analytical results for support-vector machines.

I think yummyfajita's point is that a traditional statistics approach begins with some understanding of the system being modeled, and that you create a model using that understanding. There is usually a high focus on parsimoniousness and explainability, while in ML/AI, you don't really care what the underlying model is or how the model comes to a particular conclusion. The focus is on accuracy at the expense of explainability.

I am afraid you misunderstood. OP's point isn't that it's unsupervised learning, hence not ML. His point is that it's no learning at all. IMHO.

Reply on news.ycombinator.com