How the US Air Force Ditched the Average and Saved Lives
mannhowie.com
mannhowie.com
And in general people are really bad at thinking in term of distributions. If you discuss averages or percentiles, people identify to that metric like if it applied to every individual of that distribution. That makes the debate on D&I particularly unproductive.
As of the 18th century, about 30 per cent of people died before reaching adulthood, but this percentage seems to be much higher in urban areas, which acted like population sinks. (Pathogens were really concentrated there.) Once you lived to be 20, you had more than even chance to live to 40, and a good (AFAIK over 35 per cent) chance to live to 60, but the drop-off after that was steep, 70 y.o.s were already uncommon and 80 y.o.s very rare.
Interestingly, cardinals and popes lived significantly longer, 70-somethings were a common sight in conclaves. Easier life, no military threat, good water, almost no risk of famines.
I am writing this from the top of my head, so precise values may differ from my handwaving. But I believe that the values are roughly correct.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2672390/
There was actually a major, severe DECLINE after that associated with globalization, trade, and further industrialization, which is where many charts begin.
Think about how incredible that is - nothing we would recognize as modern medicine, but adults living just as long. I can’t think of any greater indictment of our health in modern civilization, especially given the amount of resources we expend. I mean, just think about - aside from the improvements in child mortality which are almost entirely just from now nearly century old antibiotics and vaccines technology, we spend 20% of our GDP to achieve the same results as the 1850s English who got them for next to nothing in comparison. Progress!
I have to assume that our lives are overall much easier medically, though. Breaking a limb (or even losing one) is not nearly as big of a deal now than it probably was then, for instance.
These two things helped me to understand probability better.
The Signal and the Noise:
https://www.amazon.com/Signal-Noise-Many-Predictions-Fail-bu...
The videos for Harvard Statistics 110:
https://ourworldindata.org/uploads/2013/05/Life-expectancy-b...
Another very powerful demographic chart that I lost (in case someone knows where it is) is a 3 dimensional chart. Y axis is children per women, X axis is infant mortality, and then there is a slider to look at the evolution through time. All countries are plotted as points.
The reason it is powerful is because by moving the slider you can see the evolution through time. For most now developed countries, infant mortality and children per women both reduced simultaneously as the progress of medicine and the evolution of society have been very gradual. But african countries are following a different path, where the infant mortality collapsed but is not followed by a reduction of children per woman (only a few countries have started to take that turn). That is basically all you need to know to explain the major demographic shift we are about to observe in Africa.
Antibiotics are a lifesaver to reckon with.
If you were a man. If you were a woman, there was still the high risk of maternal mortality. Add to that the large number of births, and a significant amount of women died during pregnancy/birth/post-birth.
We see this immediately and intuitively from the great conditional graph in this comment: https://news.ycombinator.com/item?id=31958102
Looks right to me! Although interestingly it’s unclear which book / Book you’re referring to, as the expression comes from Exodus 2:22:
“And she bare him a son, and he called his name Gershom: for he said, I have been a stranger in a strange land.”
Personaly ALL averages should list all three averages for context and clarity as they do offer a greater insight.
example:
data 1, 2, 2, 2, 10
Mean: 3.4 Median: 2 Mode: 2
Nice online tool to easily work these out https://www.calculator.net/mean-median-mode-range-calculator...
Of note I have no idea what the average family size is children wise, just going on longstanding data from UK that in itself may be out of date, though does highlight the point.
[EDIT fixed typo]
What matters in real life is generally not how often you are correct, but rather by how much you are wrong. What you need to minimise is not your error rate, but the consequences when you are wrong. You are free to make an infinite amount of mistakes, as long as you are sure to make them in such a way that they are comparatively insignificant.
Not sure were you went there, but the point is that you can overdetail in a way that abstracts from reality.
The mean only has meaning for normal distribution, so it's my least favorite summary, because it assumes the most. Imho the normal dist is assumed way too often.
Damore included a graph of overlapping bell curves in his document to illustrate this point. Yet quite a lot of critiques I saw don't seem to understand that.
Look at the male/female height graphs on https://www.usablestats.com/lessons/normal for example (among the first examples that google came up with).
The thing that stands out is the difference between the bell curves - the area under the male height curve that is not under the female one. (Indeed, on this page, the way they drew their histogram version of the curve they explicitly drew attention to this area)
But this area isn’t representative of a meaningful population.
It’s just the sum of the excess number of men of a given height over and above the number of women of the same height.
Crucially, the vast majority of men accounted for within that population are still shorter than some women.
Unless you are, according to these numbers, over 77” tall - ie, over 6’5” - then there exist women who are taller than you.
Admittedly the population of people taller than you certainly skews heavily male - but for any randomly selected group of men, in most cases it is possible to find a group of just as many women who are all taller.
I personally find that the overlapping bell curve illustration obscures that understanding, making it emphasize more that a small number of below-average men are still taller than some women, and completely hiding the tail of outlier men on the left who are shorter than the vast majority of women…
Stacked histograms are a better way to visualize this kind of faceted distribution - but even that has issues.
It's not obvious to me. Yes the height of the male graph is lower, but is spread wider.
> It’s just the sum of the excess number of men of a given height over and above the number of women of the same height.
It's simply that the female histogram covers part of the male histogram. It's a bit confusing. If I make the graph I will colour the overlapped parts a different colour.
This is not a problem when we show only the bell curves.
Maybe the question should be "If I lived past 1 years old what is my average life expectancy?"
There are plenty of examples of where the mean is not representative of the sample and where mode or median would be more realistic, particularly where some large outliers in a relatively small sample would skew the mean.
This is one argument for homeschooling, I guess.
Teachers are generally pretty progressive (both in the political and innovation senses), but the legislation (like Common Core), state & district educational standards, and school boards move more slowly. And in the US, the primary schools are largely funded and overseen by local parents (municipal and state), so the school priorities and cultures partially reflect local priorities.
There's often debate between what individual teachers want to teach (a product of their individual preferences and backgrounds and values) vs what the administration forces them to teach (as a reflection of the political and business realities of their particular school). There's often a personality difference between the two classes too (educators vs admin types), with the former often being (at a stereotype) starry-eyed idealists and the latter being grounded management types with an eye on the numbers of finance and politics; on top of that, there is also often a labor vs mgmt (not quite "owners") divide, with individual teachers being pretty much powerless to teachers' unions being really strong in some districts. Long story short, teachers can't unilaterally change the curricula they teach.
The power struggles affect everything from math (new math, math wars, whether to test it, etc.) to social studies (CRT, Native studies, bilingualism, religious studies) to science (evolution, climate) to business needs (cursive vs typing, algorithms or letterheads), etc.
Far from being a settled matter, our educational system is heavily political and its lack of progress is probably reflective of our divisiveness as a country, where two broad sides keep playing tug-of-war and veering towards the extremes, making tiny gains and losses back and forth.
The kids end up caught in the crossfire. The US, for all the money we pour into education, ends up rather poorly educating its children vs other developed nations: https://en.wikipedia.org/wiki/Programme_for_International_St...
So for example, the average time to review a code change is not the same as the time it takes to review a code change of average size.
The average user experience of a website is not given by the user experience of the average response time.
And so on. People tend to compute the average of an easier variable and then forget that the derived value of interest is often nonlinear in the easier variable.
Roughly translated, it goes as follows:
"Following the current statistics, it turns out you're expected to be able to eat one chicken per year, and, if you can't afford it, it goes into the average anyway, because there's someone else out there eating two".
There are two pieces of bread. You eat two. I eat none. Average consumption: one bread per person
https://www.penguin.com.au/books/the-end-of-average-97801419...
resembles Dana Scully of The X-Files.
I also enjoyed Todd Rose's book, which examines the history of (mis-)interpretations and (mis-)uses of the "average":
The End of Average
https://www.amazon.com/End-Average-Unlocking-Potential-Embra...
(I am not an expert on anything.)
The average is a waste of time
I read ~10 two star reviews, ~10 four star reviews, much more depending on price. One stars tend to be full of stuff like "product hasn't arrived", "garboage", user errors. Five stars are plagued by spam and paid reviews, and also a lack of details. Three stars seem to be split between very picky buyers (esp. hate the "I paid $3.50 for this and it's not the omega particle!") and people afraid to leave lower ratings for reasons.
Definitely must read a decent sampling to see if there's a common problem, or it's mostly just malcontents, delivery issues, or misunderstanding the product or instructions, or just random issues. I've found in doing this that I'm usually surprised on the good side.
IIRC IMDB shows mean ratings, but Rotten Tomatoes shows the % of ratings above a certain threshold. I find the latter more useful.
Learning about the various types of standard control charts is quite useful for data-based decision making and monitoring the health of a process.
Control charts and Pareto chart of root causes of defects are two of the basic tools one needs to establish 'basic stability' in a process. Basic stability is a requirement before working on improving a process.
This is related to the "curse of dimensionality" https://en.m.wikipedia.org/wiki/Curse_of_dimensionality
For other uses, it can be very misleading.
Sum up is the problem.
That probably works well for average use cases, but not for the outliers.
The great thing about ICE vehicles is you generally don't have to worry about non-average use cases. To reach parity, EVs will need to be chargable in 5 minutes from empty to achieve 500 miles of range, at 5 degrees F, with 5 year old batteries. If you live in California, that may seem like an extreme use case, but it's pretty normal around here.
Similar to those electric scooters in Asia, Just imagine if you rolled over a pitstop and had the battery quicky taken out from below the car and then a new battery placed back in.
That sort of system could change the economics of building an ev as you could move to a model where users don't really own the battery they rent it for a while until they need a swap.
Or electric cargo bicycles. Because every bike on the road is one less massive car.
Sure it can work, but overall I have to call it a bad compromise.
Your first idea requires new, expensive infrastructure that will need maintenance due to mechanical parts. Given charging points are regularly broken at the moment, I don’t have high hopes. But even if we overcame that: it’s impractical in almost every major city in the World, because you’d have to drastically alter existing spaces, such as filling stations.
This might work in your context, and many others. It might even work for the “average” use case. But for 75% of the developed World (where the EV market is the fastest growing), I’m not convinced.
And the electric cargo bike argument - or bikes in general - work great for the “average” journey, but not many people actually take that journey.
My most frequent drive is 200+ miles in normally wet or cold weather. My next most frequent is 15+ miles in baking hot Summer sunshine in a heavily congested city where I need aircon. Neither would be pleasant on a bike - electric or not.
And I think that’s the point of this discussion: engineering for averages is a terrible idea, and most EVs don’t work for most journey profiles for most of the World’s drivers today, compared to ICE
One problem was that they had chosen one model of car to simplify battery inventory, but the car was not a good fit for their initial markets.
But I think the main complication comes to ownership. Are you going to own an EV and an ICE? Or are you doing to own and EV and rent an ICE when needed? Or the other way around? And even if you rent it can become complicated. Last weekend I rented a car and it needed to be an EV since I booked it late and that's all that was late. I personally would rather not have dealt with the extra complexity of charging it compared to just putting gas in the tank.
But really I thin the issue is that if we move to EVs it kind of forces some sort of vehicle specialization where it previously didn't feel as necessary. That opens a big can of worms when it comes to changes in lifestyle for many people.
They do when ICEs are getting banned.
Driving from San Diego to Sacramento is about 800km; in good weather and light traffic you could cover that in 7 or 8 hours. Get up early, drive that first leg, stop for lunch, then keep going (e.g. towards Portland or Seattle).
Of course, an American would usually stop for gas before hitting empty because we're used to the idea of "No food or fuel next 2 hours".
Took 2-3 days. He slept a few hours in the car at night.
Yes, that is not an average situation, but there's a lot of peace of mind not having to think about it at all. By the time I get down to 250 miles in my ICE, I can fill up in 5 minutes, but still have a very comfortable margin for anything that may happen.
And, worst case, if I run out of gas (never happened in my lifetime)? I can get a gallon of fuel and drive away. How does that work with an EV?
https://www.vox.com/identities/2017/8/8/16106728/google-fire...
Because certain people constantly use the fact that there isn't 50/50 gender representation in engineering roles as a way to imply the profession is particularly sexist. It was absolutely relevant to the conversation at hand around "women in stem".
That's not what I remember it saying. Are you sure you're remembering it right?
> Openness directed towards feelings and aesthetics rather than ideas. Women generally also have a stronger interest in people rather than things, relative to men (also interpreted as empathizing vs. systemizing).
> These two differences in part explain why women relatively prefer jobs in social or artistic areas. More men may like coding because it requires systemizing and even within SWEs, comparatively more women work on front end, which deals with both people and aesthetics.
https://gizmodo.com/exclusive-heres-the-full-10-page-anti-di...
Puh-lease
Assuming normal distribution:
Single tail 4s.d. is like 1 in 30k. There are about 300M people in the US. Doing the division, that's 10k people.
Google employs way more software engineers than that.
---
Google does hire at the tail (just not as extreme as you claim). And when you select for the tail, slight difference in average translates to a huge gender imbalance.
Take height for example, we have quite some overlap between men and women. But when we select for people over 6', it's gonna be overwhelmingly men.
Can't comment on intelligence distributions.
To talk about whether intelligence follows a (one-dimensional) normal distribution we have to assign a number to it. That number is usually IQ, but by design the raw score is transformed to make IQ scores follow a normal distribution.
So it is trivially true.
If we want to go beyond that, what does it even mean to say, for example, "twice as smart"?
Hence the name of a rather notorious book by Charles Murray.