Mercury is the closest planet on average, but calling it the closest neighbor is just confusing.
When I think of 'neighbors' in the context of the solar system, I am generally thinking of neighboring orbits. It would be hard to argue that Venus's orbit is not the closest orbit to Earth's. Or at least it would seem silly to do so.
Maybe I'm being overly pedantic here, but my view is that orbits have neighbors, and planets have orbits, but planets don't necessarily have neighbors. Something in the word 'neighbor' implies persistence to me, so I don't really consider average-closest planet to be a neighbor when exactly which planet is closest to any other changes constantly.
https://www.merriam-webster.com/dictionary/neighbor
So I guess that means venus's orbit is our closest neighbour, but not the planet itself most of the time....
That's why we say "next door neighbour"