Sets and Probability | Hacker News Reader

Sets and Probability

stopa.io

96 pointsstopachka35 comments

Probability theory with sets actually generalizes well to infinite and even uncountable cardinalities—rather than counting events you just have to switch to the more general notion of the measure of a (sub)set [1].

[1] https://en.wikipedia.org/wiki/Measure_(mathematics)

For anyone looking properly (i.e. with proofs) learn measure theory, I highly recommend "Introduction to Measure Theory" [1] by Terrence Tao (has an excellent collection of exercises -- they're crucial to the reading!)

I think it's OK to skim the first few chapters about constructing Lebesgue measure from Jordan measure, as it makes the whole topic appear more difficult than it needs to be. From Chapter 4 (abstract measure spaces) onwards, the book is full of great insight.

A good one on probability & stochastic processes is "Measure Theory and Probability Theory" by Athreya & Lahiri.

[1] https://terrytao.files.wordpress.com/2011/01/measure-book1.p... [2] https://www.springer.com/gp/book/9780387329031

How useful do you think measure theory is from a practical perspective? I've seemed to get by without more or less during my career (which is pretty stats heavy).

Learning measure theory has eliminated a lot of blind spots for me when it comes to stats / probability / ML. I find that I get "stuck" much less frequently when reading papers, and I'm able to better in the gaps when authors are too sparse with the mathematical details. Before, I would always wonder, "how on earth did they come up with this idea?!", but the more math I learn, the more I'm able to recognize that certain ideas are inspired by XYZ branch of math.

It's also been useful for understanding the "novelty" papers like "Neural Differential Equations" and so on.

I was lucky enough to take a good measure theory class in college, where it was fine to spend an entire day working through a proof from a textbook. Now that I'm working, though, it would be harder to justify that time commitment. So it really depends on your priorities.

I’d just make a slight correction in case anyone was confused. The book actually only has two chapters, one on the Lebesgue Measure and one on Abstract Measures. When the person above me said skim the first few chapters they meant sections of the first chapter. I’ll admit it’s a somewhat confusing way to lay out the book.

I also can’t recommend this text enough for anyone looking to learn measure theory. Tao is not only one of the brightest research mathematicians but also is extremely good at communicating and framing concepts.

One of my favorite books Epistemology and Psychology of Human Judgment by Bishop & Trout [0] argues that humans are bad at making judgments, and what we want is reliable methods for arriving at truth.

The authors of the book provide several heuristics that yield better outcomes than typical strategies people use.

With respect to probabilities, since humans, on average, suck at it, they recommend a "frequentist" approach. A great illustration is one about a 99% accurate medical test that claims a patient is positive. The probability the patient has the disease is not 99% (over 60% of Harvard doctors get this question wrong). The authors show how reframing the question with a sample population (10,000 people for example) and the relative prevalence of the disease (and noticing the false negatives) the problem becomes almost trivial to calculate.

[0] https://www.amazon.com/Epistemology-Psychology-Judgment-Mich...

note, folks the book can also be found on archive.org as well

The set-based intuitions are a nice guide, but knowing how to turn the crank and churn out the calculations is also crucial for the cases where intuitive thinking doesn't scale.

yeah, I also liked this guide. Would love more articles about probabilities (but i always loved this topic :)), there's only one exception: the "3 door problem" (Monty Hall problem). In my opinion the general misconception about this problem does not stem from a misunderstanding of probabilites but a misconception about the moderator (who plays by his own rules [as a bad actor] - which is often not really discussed although it changes the whole game). Well, I would read an article about it by this author anyway ;)

Not sure what you mean with "bad actor" - the host/moderator is not an adversary, his rules are fixed from the outset (see e.g. Standard Assuptions in [1])

These rules imply that new information is revealed, and based on the new information it is advantageous to switch.

The trickyness of the Monty Hall Problem lies in identifying that indeed new information has been revealed, which seems counter-intuitive.

[1] https://en.wikipedia.org/wiki/Monty_Hall_problem

Good on wikipedia for having them, but in my experience those "standard assumptions" are often left out when stating the problem.

I think the assumption that tricks most people is that the host will never eliminate the door with the prize. Since this makes sense for a real TV show (it would kind of crappy from the host to eliminate the door with the prize) sometimes it goes unmentioned.

But, even if you highlight the assumption, for most people it is not straightforward how to use this information.

Not really in my experience, I've never come across the problem without being stated explicitly how the moderator acts - it would be a different problem.

What I think is true is that people underestimate the role of the host (i.e. the rules according to which he behaves to).

The first statement of the problem in the wikipedia page (from "Ask Marilyn") does not state it explicitly (though it does say the host knows what's behind the doors). There is a reason wikipedia refers to them as "assumptions".

Or from [0]: "The standard annunciation of the MH problem, does not make explicit what I am assuming here: namely, that Monty will always open a door which does not contain a prize."

[0]: https://www.montyhallproblem.com/#F2

While you are technically correct, it's implicitly stated ("game show") and correctly understood by the majority of readers. Take it from Marilyn vos Savant herself:

"Virtually all of my critics understood the intended scenario. I personally read nearly three thousand letters (out of the many additional thousands that arrived) and found nearly every one insisting simply that because two options remained (or an equivalent error), the chances were even. Very few raised questions about ambiguity, and the letters actually published in the column were not among those few." [0]

The crux of the problem is the counter-intuitive nature of probabilities and information, possible choices and choices that could have been. It's a difficult problem, purported to have tripped up even Paul Erdős.

[0] https://en.wikipedia.org/wiki/Monty_Hall_problem#Other_host_...

> In my opinion the general misconception about this problem does not stem from a misunderstanding of probabilites but a misconception about the moderator.

An analysis of the replies to the column in which the puzzle first had wide circulation showed that most of the respondents disputing the correct answer had understood the problem as intended, but still got the probabilities wrong. At that time, the game show on which it was based was well-known, so confusion over the moderator's role (which is not as a "bad actor") was less likely than now.

> Would love more articles about probabilities...

Well, I wrote a few articles on the topic from a Bayesian perspective few years ago:

http://blog.moertel.com/tags/probability.html

(Only one of them is about the Monty Hall problem.)

Using the counting method for the Monty Hall problem actually works very well. Initially there are 3 possible ways the goats and treasure can be arranged:

goat goat treasure

goat treasure goat

treasure goat goat

You pick a door -- let's say it's the first door. Monty opens one of the other 2 to reveal a goat, leaving one other door. This modifies the possible outcomes:

goat treasure

treasure goat

Using the counting approach, there is 1 outcome where door 1 has the treasure, and 2 outcomes where the other door has the treasure. 2/3 chance of finding the treasure if you switch.

It's indeed a nice guide. One place where the intuition stops unexpectedly, is calculating the probability of getting 2 reds balls out (without replacement). In the situation with all balls together, the chance would be 1/4. But with two separate boxes, the probability would change to 4999/20000 (1/2 * (1/100 * (0 + 1/2 * 99/100) + 99/100 * (1/2 * 1/100 + 1/2 * 98/99))).

There's no 1/4 here. (1/2) (99/199) = 99/398 First ball removed changes the odds for second draw.

It's really not just a guide. It is practically the basis. Yes, knowing how to do the calculations will make life much simpler, but I often find the usual calculation rules are easy to misuse. I often derive things from scratch using the set based axioms.

I'm bit confused, isn't counting desirable outcomes vs possible outcomes standard way of teaching probability? How is nCr/nPr stuff explained if not in the context of counting outcomes?

I'm also confused where does set theory come in here?

To begin, Sample Space of Events is a Set.

Simple cases like probability of A or A's complement etc., all rely on basic set theory.

The rabbit hole is sample spaces that are really really large.

What happens to probability distribution of desired events in those cases?

This method only works because the boxes have the same number of balls. To calculate probabilities by counting outcomes you have to start with outcomes that all have equal probabilities.

It's a special-case of a general principle that works regardless of the weighting. Instead of integer counting, do real counting with unity.

Sure you can generalize it by weighting the outcomes by their probabilities (not mentioned in the article). But how are you going to calculate those probabilities any more easily than you can solve the posed problem?

The more "intuitive" way of generalizing it (integer counting) does not work in general. I think the example is likely to be misleading to anyone who doesn't already understand this stuff.

As suggested by the footnote, the reason the given example can be solved elegantly is because of the symmetry.

Ironically, this is the sort of probability theory Nassim Taleb hates. :-)

Exactly. This is the brand of neutered, ludic probability on which The Black Swan is a holistic dump. Very ironic indeed that it leads with Taleb as the inspiration.

(From the name of OP I deduce he may read this.)

You should read the first and second chapter of 'Probability Theory: the Logic of Science' by Jaynes. It offers a derivation of probability theory using only Logic and Boolean algebra as foundations. I found it delightful, I hope you will also enjoy it :)

It's somewhat more intensive than Taleb, but the derivations are really beautiful.

Will do, thank you! : }

I don't recall how I learned it in high school, but for my undergrad statistics & probability course, sets were very much the basis for everything. I'm surprised that this isn't the norm ...?

This was so satisfying to read.

Thank you for the kind words :)

References to textbooks that OP read would be a nice addition to the essay

My favorite was this textbook:

https://www.amazon.com/gp/product/1292025042/ref=ox_sc_act_t...

This one was more dense, but very good in the beginning if you take the time: https://www.amazon.com/gp/product/0070484686/ref=ppx_od_dt_b...

---

Will update post later, but wanted to share with you now : } -- thanks for taking the time to read!

Reply on news.ycombinator.com