These things have become so esoteric and expensive because the mechanical keyboard hobby is undergoing a more intense version of what happens to most hobbies/pastimes; the purpose of the hobby has transitioned from primarily "making/using the thing" to "obtaining and displaying ever more extreme versions of the thing to impress other people who know about the thing".

I feel like this is an outgrowth of the basic human desire to form groups and define an in-group and an out-group; people start a hobby because they are interested in whatever, but after a while there develops an (often arbitrary) right way and wrong way to pursue the hobby, cliques form, and after a while participating in the hobby becomes an exercise in demonstrating that you've bought/made/done the right things and are therefore worthy of being in the in-group instead of the out-group.

I think this is so strong in the mechanical keyboard community because there's not a lot of "there" there; mechanical keyboards are nice to type on, but there's no core competitive activity that they're used for - you're not spending $1600 on a keyboard to improve your typing speed and accuracy 1.2% so you can have a shot at winning the big annual World Typing Tournament in Las Vegas. The mechanical keyboard community feels to me more like the streetwear or sneakerhead community than, say, R/C planes or surfing or whatever.