The difference between the middle class and upper middle class is infinitesimally small compared to the difference between the upper class and everyone else. Your argument is what's been commonly used lately to dupe people into thinking their doctor neighbor is the enemy rather than the guy who owns 2 islands and 4 yachts.
Want to know what affects housing prices a whole lot more than one person buying one house? An investor with dollar signs in his eyes buying up 20 of them.