Possible improvement: The median will always be a member of the actual dataset, so after calculating the mean, run through the data to find the entry which is closest to it, and return that for the "median". Could make our dastardly plan harder to uncover.
- Any value that minimizes the sum of absolute differences? Then all real numbers between 3 and 5 are a median of that set.
- With an inequality over the PDF? The same as above.
- The 1/2-quantile? Then you need to tell me if we're rounding up or down.
I've seen that definition before, but it's a completely useless definition for any legitimate use case of the median as an order statistic. I have no idea where it comes from.
For example, names can be ordered alphabetically, so you can find the median of a set of names! OK, that's a silly example, but I expect someone could come up with a more sensible one.
Another case might be where you're using geometric means everywhere else in your working, but then you want a median in one place, but there are an even number of elements, so why would you suddenly use an arithmetic mean of the two middle elements?
- Imagine you hand out a survey where some answers are on a likert scale. "strongly dislike" is worse than "dislike", which is worse than "neutral" and so on.
- The median ranking over a set of ranking on different criteria.
- When the domain is not the real numbers. "The median employee of this company has 1.5 kids".
It's also the definition that I was taught at school.
Oxford Languages defines it as: "denoting or relating to a value or quantity lying at the midpoint of a frequency distribution of observed values or quantities, such that there is an equal probability of falling above or below it", which also supports the idea of taking the midpoint between the two middle values.
Merriam Webster define it explicitly in the way OP does (arithmetic mean of the two middle values): https://www.merriam-webster.com/dictionary/median
I'm not sure why you think it's a 'completely useless definition'. It would perhaps be a logical argument that data sets of even length do not have a median, but I think this would be even more useless.
To the layperson (which Mary Jones undoubtedly is), providing the 'median' age as an age half of the employees are older than/younger than, is what would be expected. Your first bullet point makes sense here. For example here:
18, 20, 25, 30, [35, 40], 45, 50, 55, 60
Most people would expect the median to be calculated as 37.5. However, as you mention, any real number higher than 35 and lower than 40 would equally fulfill the definition of 'equal probability of falling above or below' assuming this is the full population. Would you then argue that there are infinitely many medians in this data set, or none?
There are other issues with calculating 37.5, such as the fact it implies a higher degree of precision than it ought to. But ultimately, as a practical and usable insight into data, especially data with extreme outliers or high levels of skew, I strongly disagree that this definition is completely useless.
By the definition I usually go by (the one with the inequalities), that dataset has infinitely many medians. There is no reason to say the midpoint is privileged. This is a better definition because it also works on sets where an order is defined but the arithmetic mean is not.
If asked to report a single value with no qualifications, I would report either extreme of the interval.
This is a matter of opinion. I agree it's not 'privileged', but it sacrifices a minimal amount of accuracy for a usable answer.
> If asked to report a single value with no qualifications, I would report either extreme of the interval.
That's a very interesting answer - so for my contrived example of 18, 20, 25, 30, [35, 40], 45, 50, 55, 60, you would report 35 or 40?
How about if I change the dataset a bit (so it's even more contrived): 18, 18, 18, 18, [18, 60], 60, 60, 60, 60 - would you still report either 18 or 60 (again, asking without qualifications)?
I think most people would agree that the median calculated as the arithmetic mean of the two middle values (39) is misleading, but that it is less misleading than ignoring half of the data set. How would you respond to that?
Generally I report the lower end because it more easily generalizes to different quantiles.
> How about if I change the dataset a bit would you still report either 18 or 60 (again, asking without qualifications)?
Yes. I don't see how answering 39 is any better in that case.
> How would you respond to that?
I would respond that you should look at the distribution. Reporting the median is particularly misleading in the case of bimodal distributions. I would try to explain that summary statistics are to be taken with caution, possibly showing a picture.
Fair enough. I think 39 would show there's no bias towards younger or older employees. I think 39 is better but I can see both sides of the argument, and can ultimately see that it boils down to a question of personal preference as to which of two wrong answers is 'less wrong'.
I went to archive.org and searched for intro books in statistics (there are a lot!). All of them gave preference to the average of the two middle values. One of them also pointed out there were valid alternatives.
"Since there are 10 values [in the sorted list], the median is the average of the fifth and sixth values. However, it could be also be 4.1, or 4.9, or any other value between 4 or 5, since each of these values divides the 10 values into two equal groups." - https://archive.org/details/introductiontost0000chao/page/54...
"For a sample of size n, the sample median is the ... Average of the two order statistics in the middle if n is even." - https://archive.org/details/introductiontost0000salv/page/64...
"We find no middle value; therefore, the median is the mean of the two middle values" - https://archive.org/details/US_Navy_Training_Course_-_Introd...
"The sample median for this collection is given by the middle observation if n is odd. If n is even, the sample median is the average of the two middle observations." - https://archive.org/details/introductiontost0000milt_g9m2/pa...
"sample median = ... the average of the middle two values if n is even" - https://archive.org/details/introductiontost0000peck_o4j9/pa...
"If there were an even number of observations (say, six instead of five), the calculation would be more complex. We would need to find the middle pair of numbers (the third and fourth observations), and then find the value that’s halfway between them, by adding the values together and dividing by two." - https://archive.org/details/introductiontost0000haan/page/n6...
"If there is an even number of values, the median is equal to the mean of the two middle values of the ordered set." - https://archive.org/details/introductiontoec0000merr/page/18...
1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,10^30
I read …, 1, 10, 30 but …, 1, 10^30 is far more dramatic.
| u - m | = |E(X - m)|
<= E(|X - m|)
<= E(|X - u|)
<= E(sqrt((X - u)^2))
<= stddev
fit N(u, s) to {(x, y)...} then m will lie u+-s
rather than, def. s^2 = E[X^2] - E[X]^2