Hacker News Reader

Comment by zozbot234 | Hacker News Reader

Stats requires calculus; you can't even define a probability distribution without some pretty advanced notions of real analysis. Discrete math, linear algebra etc. are viable alternatives.

View on HN

Stats requires calculus; you can't even define a probability distribution without some pretty advanced notions of real analysis.

Only in the same sense in which operating a car requires advanced knowledge of mechanical engineering, electrical engineering, aerodynamics, fluid dynamics, thermodynamics, etc. IOW, in no sense related to the everyday, ordinary practice of operating a car.

Sure, statistics requires calculus... and real analysis, and probability, and measure theory, and FSM knows what else - IF you're doing statistics research or trying to break new ground, or do really advanced things. But all of this is light years away from the level of statistics knowledge needed by Joe Q. Public to better understand (and not be misled by) the "statistics" frequently thrown out in news articles, government reports, etc.

Please, for the love of FSM, can we stop this HN "thing" of assuming that every mention of any mathematical topic implies that the goal of the user/learner is to do original research in the field?

Is this really true, in practice? For example, given its importance in every day life, I think everyone should understand test sensitivity and specificity, and how these relate to but are quite different from predictive value positive and predictive value negative. All of those topics can be understood with basic algebra. Similarly, I recall my introduction to biostatistics class I took at an Ivy League institution, and I don't really recall using calculus much in any of it.

At some point, someone was saying precisely this about literacy. And it was as true then as your comment is now. You only need some range of function around the baseline education to operate in society. But if you want society to progress, then everyone needs to learn to read and write.

Bad analogy. Being able to understand statistics at an everyday usage level is the equivalent of being able to read and write. Understanding calculus needed for statistics is the equivalent of needing to understand Latin or comparative linguistics. Noble and useful pursuits for sure, but you can get far with an educated society that just knows how to read and write even if they're iffy on the Latin roots of it all.

This is quite clearly false. Most people on the planet have zero to little understanding of statistics. They get by just fine. The field of Calculus is actually fairly simple compared to stats. By and large calculus is the study of the slope of a line, and the area under the curve.

Statistics is more complicated philosophically and computationally, as you do in fact need calculus to do statistics, since a fundamental tool of statistics is evaluating the area under a variety of distribution curves.

You've misunderstood my post. I'm not arguing that needing to understand statistics is the same as needing to be able to read and write; I was responding to a post that was arguing that teaching stats without the underpinnings of calculus is like a society that is barely literate.

> Statistics is more complicated philosophically and computationally, as you do in fact need calculus to do statistics

This is just fundamentally false, as there are many statistical concepts and introductory statistics courses (given at some of the most renowned universities in the world) that can be taught without an underpinning of calculus. It's like arguing that you can't teach a programming class until someone understands digital logic design.

Again, philosophically and computationally, statistics is more complicated.

And again, you do need calculus to do statistics. Maybe you're saying: "If someone does the calculus part for you, you can use this one equation to solve this one question."

That's akin to calling a spelling test literature. You are confusing practice with knowledge. Anyone can bang on a drum. That does not make them a drummer.

At some point someone was saying the same thing about Greek and cursive, too, though, so being merely more than the baseline isn't sufficient evidence that it should be included in everyone's education.

Not at a high school level. AP Stats I believe does not require calculus as a prerequisite.

Discrete distributions can be defined, even infinite discrete distributions (as sequences and series are taught in pre-calc). Continuous distributions can't be formally defined, but a lot of intuition can be given with hand-waving e.g. the area under this curve. Probably most of the class is spent with counting and probability-type problems, but plenty of actual statistics can be done without calculus - we can learn about distributions, what a statistic is, expectation, sampling, counting, probability... the list goes on.

But that's like saying everybody who learns to program needs to know electrical engineering so they understand how CPUs work. Just as most programmers don't need that to be good programmers, knowing useful statistics doesn't mean deriving things from first principles but rather knowing what statistical test to apply to analyze data and how to interpret the results. This doesn't need calculus.

I took AP Stats in high school. I took a calculus based probability course at MIT. The former was extremely important to me and I learned a ton. The latter was interesting, but mostly unnecessary. A z-score lookup table is more than enough to teach the concept of a normal cdf without actually being able to derive it yourself.

As a professional data scientist I've never needed to use calculus unless you consider graphical reasoning on distribution diagrams to be calculus.

What you say is true. However:

Most people who deal with data/statistics in their working lives do not need to know real analysis, do not utilize calculus to draw (correct) conclusions, and do not use linear algebra either. They learned all these things once and forgot them a long time ago, because they did not need them. They utilize statistics an order of a magnitude more often than they do calculus.

They are not statisticians, but people who need to deal with data as part of their job.

No. Learning high-school or early-ungrad statistics does not require knowing calculus. The material will not require integration or differentiation.

But statistical literacy doesn’t.

Sure, they may not be able to tell you what is a measurable function, but they can explain the way you should feel about a p-value.

As someone who took both a stats and newtonian physics course before taking a calc course, I wish I hadn't. It was a waste of time. They can't explain why you use the formulas they do, they have to just say "trust us, and be able to regurgitate it on the exam". For me, learning means developing an intuition, which means resolving/building new facts from other facts I have already accepted. Being handed seemingly random formulas to memorize goes directly against this. Yeah, I can use my car without knowing how every piece inside works, but the moment something goes wrong, I don't know what to do. I would never say I am car-literate, and someone who hasn't taken calc cannot be stats-literate.

You can develop an intuition divorced from meaningless formulas.

It goes without saying this proposed class wouldn’t be based around memorizing unmotivated formulas.

But how would you "motivate" the formulas without knowing where they came from? Why is this the formula we use and not something else?

To go back to the car analogy, I know why I need an engine, you might even say I know how to use the engine, but if the engine dies or I want to use the engine for some other purpose, I'm not equipped to do anything.

I don't have "literacy" with engines, I have rote memorization of a series of steps. I don't have enough information to know why the steps are what they are, nor could I know under what conditions the steps should change or what they should change to.

Just don’t include formulas you can’t motivate simply.

Will it result in students not knowing as many formulas? Of course, but who cares?

Sometimes they'll need formulas they don't understand (can't motivate) for doing work things?

Then what? Are they going to incorrectly use anther one, a wrong one, because they understand it better?

(Searching not where one lost the key, but where the light is stronger)

Hmm, I wonder if you studied stats? It sounds as if you studied lots of math.

I did long ago, mostly forgotten, and, hmm, I don't think there's a single real life useful probability function one can understand, without knowing the underlying calculus? Except for understanding in the sense of trusting the teacher.

Which works pretty ok in medicine

You can't even define arithmetic on natural numbers without some pretty advanced notions of logic and set theory. But you can definitely get a lot of value from arithmetic without those definitions.

Colleges have varying levels of stats classes for different majors, and not all of them require rigorous understandings of calculus. The ideas and principles presented in those classes are still important for people to learn, even if they are unfamiliar with, or haven't mastered, calculus. It's possible to teach those principles in high school, as well.

Then I have no idea what my school was doing then because as a senior you either took Calc or Stats.

There’s a diminishing return here. Single variable differential and integral calc is a sweet spot. Without that, there’s a ton of memorizing seemingly unrelated facts, but with it, you can learn a few principles that lead to a huge amount of practical stuff.

i’d argue that you can’t properly define probability without notions in measure theory, which is obviously far too advanced for a high school student. i’m not an educator, but some middle ground needs to be struck. i think it’s clear to many that the quality of education in american colleges far exceeds the quality of education in the average middle or high school. that’s the issue, imo

You can do quite a lot of useful stuff with just https://en.wikipedia.org/wiki/Cumulative_distribution_functi...

You only need measure theory when working with something that is not easily replaceable by R^n, Z^n, or finite sets to meaningfully define integration, otherwise (in)finite sums and Riemann integration get you very far.

I am a bit rusty on my advanced probability theory, but IIRC the only thing that required* measures was defining conditional probabilities and expected values on zero-probability events.

Of course redoing that class without Lebesgue integration sounds excruciatingly painful.

* Not just to make proofs nicer and theorems more powerful

Reply on news.ycombinator.com