DST has a high cost to anyone who is responsible for creatures that do not understand it - children and pets.
Noon is when the sun is 50% done with its cycle from rise to set. We base our clocks on that. Not delusion, abstraction. It is simply a measure of objective reality.
Deciding that noon is at 1:00pm on the longest days of the year so that the sun can set at night instead of evening is delusion. Deciding to make that permanent all year around isn't any better.
Time zones boil down to "for consistency's sake, we cannot have an infinitely granular way of setting time across longitudes" and people chopped the world into slices. At the middle of each slice, noon is supposed to be noon and is denoted with "12:00(pm)". This is a sensible compromise between clocks changing by the second based on GPS coordinates and the whole world having the same time. " noon is when we decide makes the most sense logistically" is not the argument here, it takes the nuance out. Noon is when, at the middle of the time zone, the sun is midway in the daylight part of the cycle. This is the crux of it, not simply "we all decided". We all decided because of something. there has to be a basis in reality, otherwise society is just shared delusion and things go off the rails quickly.
Hell, the application of 12 to high noon is relatively new. For the Romans, it was the transition between the sixth and seventh hours of the day, but they thought of it as the end of the sixth hour. Because of this, 'six/seis' is the root word of 'siesta', the mid day break in Spanish speaking countries.
Society is not going to "go off the rails quickly" as you suggest just from us moving the numerals we decided in the first place to new temporal locations, locations that we've moved them to and from twice a year for over a century.
http://blog.poormansmath.net/images/SolarTimeVsStandardTime....
But of course a whole lot of the population lives in the part where it's worse.
In addition, people who don't think it's a big deal because other people have dealt with the problems is causes for them or they have been statistically lucky in never having had a related problem.