They are insulated like a thermos (but more efficiently), and since they are really big, they have small surface area compared to their volume.
It is a VERY efficient way to store heat over shorter periods, like a week or so, and it helps making power plants more cost efficient.
The best way to make it efficient for long term storage (like seasonal storage) is to
1. Make them huge
2. Make them ball shaped
3. Put them underground
There has been talk of tryibg this out in Denmark. Almost all our powerplants (i.e. the not-really-old ones) are combined heat and power plants, and because we use district heating water to create a vacuum behind the power producing turbines turbines (instead of sea water) we go from a theoretical limit on the energy efficiency of around 60% to 98%, as we do not pump out the excess heat into the sea.
Newer combined heat and power plants have an energy efficiency of around 95% to 96%. The trade of is a slightly lower vacuum, meaning less production of power. However, we can scale up heat production to reduce costs of running power plants when energy prices are low (or even negative).