Well, yeah. You can't account for everything. Traditional sports don't consider elevation, air quality, wind, etc.
Does anyone disregard a track world record because the weather was pleasant that day, and the previous recordholder did it on a hot, humid one? Does anyone argue Barry Bonds' home run title should be disregarded because of the thin air at his away games in Denver?
Take away wind assistance, and Bucky Dent loses his place in history. Take away a pebble in the grass and the Red Sox win the 1986 World Series. Add some rain to give pitchers an extra day of rest, and all sorts of baseball history gets upended.