The real weird one from that era was the 7 segment led MHz displays that were popular in the 386/486 era. It was weird because the the chips could not change speed in the first place. And I don't want to say every system was like this, but every one I ever saw was set by a set of static jumpers or a dip switch. When you upgraded your cpu you put the little sticker that came with it on the chassis and changed the jumpers, they served the same purpose.
I do remember using it to make older games play less bad, but, as you say, this rarely worked, I think precisely because the speed was never the original one (I don't remember the clock speed of my first PC, but the first one with a turbo button ran at 33 MHz which I'm sure is a lot more than twice the speed of the first one; seems closer to 8x from what a quick search implies).
I didn’t learn the real purpose until long after they were gone.
I knew what it was for and had some old games that ran too fast, but from what I remember, in most cases 16 MHz was still much too fast, so I didn't use the button much.
I think it was no problem to switch Turbo mode on and off in the middle of running software.