Instrumenting Python GIL with eBPF

But I think an interesting one is the also literally the most common example that's in many stats textbooks, at least the ones that cover Bayesian stats. That being that if a test is 95% accurate, that it does not mean that that scoring positive on a test means that there's a 95% chance you have whatever the test is testing for.

I think there's a great irony in the latter, as I find in the tech and engineering communities a lot of confidence, especially in understanding numbers yet very often make this mistake[0], and frequently want to establish meritocracies (advocating for standardized testing and leet code). But understanding the former should result in thinking the latter is ineffective.[1]

--------------------------------

[0] I particularly like 3Blue1Brown's introduction to Bayes theorem as it includes the Kahneman and Tversky questions AND (most importantly) he discusses how when the questions are reframed that peoples accuracy can dramatically swing from overwhelmingly wrong to overwhelmingly correct. I like this because the hardest thing in doing statistics (and much of science) is actually down to this. It is all about understanding the assumptions you are making, and more specifically, the hidden ones. https://www.youtube.com/watch?v=HZGCoVF3YvM

[1] I think this one is worth working out a little and so let's apply Bayes to a proxy problem (for illustration). It's good to start to understand how metrics can undermine meritocracies and it will illustrate how people actually lie with statistics despite never saying anything that is actually untrue (and that is why it is so common and why often lying with stats/data is unintentional).

Let's let monetary wealth represent our measure of success, let's assume that college entrance exams (and the ability to graduate) is purely meritocratic (i.e. "I went to Standford" -> "I'm smart"), let's define $100m net worth as ultra wealthy, let's assume that graduation from a college gives a person just intelligence and the connections you make there are not significantly influential to success (this one is actually key, but "left to reader as an exercise"), and let's assume your final net worth is purely due to your own work/efforts. We're using this because the frequency of which awarding institution is used to imply one's value/worth/skill (your cousin/friend/boss/coworker who brags about being an "x" graduate or parents showing off their kid's intelligence, etc) and how admittance to these institutions has a very strong correlation to standardized test scores.

Top 8 schools here are Harvard, MIT, Stanford, UPenn, Columbia, Yale, Cornell, Princeton which represents 7%, 5%, 5%, 4%, 4%, 4%, 3%, and 3% of US ultra wealthy, respectively. And these represent 35% of all of the US's ultra wealthy, which there are 10,660 today (~28.5k globally). So 746, 533, 426, and 320 people, respectively. (Note we now have to assume there isn't double/triple counting: e.g. undergraduate Harvard, masters MIT, PhD Stanford) Each year these schools graduate 10k (Harvard is a bit confusing), 3.5k, 5.2k, and so on. Now I don't have data for when those people graduated from those schools, so we'll describe a function to help us estimate. Using Harvard we have 746 people / (10k graduates * number of years). I was able to find that the median age of this group is >55 but also evidence of a bimodal distribution). So let's naively assume a 10 year period, as this still gets us a pretty conservative/lower bound estimate (as well as assuming no dupes). So now we have that Harvard represents 7% of US centimillionaires, but only (and this is an absurd overestimate) 0.7% of Harvard graduates are centimillionaires.

Now we see a framing issue. Harvard tries to show its value by advertising that it is the highest producer of centimillioaires, citing this 7% number. But when we consider context (i.e. the number of graduating people) we find that far fewer than 1% of graduates make it to that level of success, which suggests that a Harvard education has little to nothing to do with said success. Even if we try a clearer cut case like with MIT where there are 2k engineering degrees awarded a year (200 arch, 100 hum/art/soc. sci, 1k management, 500 sci, etc) then we have 533/(2k * 10) or 2.6% and this still does not imply that MIT has a large effect on you becoming ultra wealthy, because this all relates to the exact same problem as the intro medical test problem: the base population size is large and the event you're testing for is rare. In fact, you can do this type of analysis for any test and mark of success and you'll probably end up with very similar results. The underlying reasoning being that there are many factors that actually lead to success and any single one of them is not a significant indicator, but it is the agglomeration of weakly influencing events that accumulate over time. Of course these numbers still have utility and this doesn't suggest you shouldn't get a degree or that you shouldn't aim for a top school degree, but they have to be understood under context and that context is often complex. Which removing the context is how one lies with statistics/data and the most common person that lie is told to is yourself.