Rush to idle is a compromise. There are two basic kinds of loads. User responsive and background tasks.
User tasks want to use the rush to idle because it improves responsiveness. Most background tasks don’t really care.
High performance cores trade power efficient for execution speed by using lots of transistors to analyze the code and optimize on the fly (literally a semi-programmable core in itself to run your core faster). In addition, power consumption is linear with clocks in theory, but not at all in practice.
Let’s consider AMD specifically. In order or semi in order cores are much narrower. All the speculation bits go away. The extra load/store complexity (very significant) also disappears because the core simply can’t use it. SMT will also go away. As it adds a ton of duplication, that also reduces size (10% if I remember Intel’s claims from a few years ago). Simple cores get much better performance per watt and per transistor, but at the expense of time. Makes a lot of sense if time isn’t critical.
On desktop, a zen 3 core at 4.5GHz will use around 12w. Dropping frequency just 200MHz will lower power usage per core by almost 2w. (from Anandtech).
In the space of that one core you mention, you could fit 4 complete in order cores with lots of room to spare. They could all run at a couple GHz and still use less than half the 3.8w you mention.