In fact, median is very very explicitly what you want to use, because outliers are just that and the vast majority of people aren't.

One study on UNC-Chapel Hill graduates in the 1980s said that the best-paying major was geography. Why? Michael Jordan majored in geography.