It is not exponentially better. Energy stored in a stacked mass scales as n^2 whereas horizontal stacks scales as n. That’s quadratic scaling, not exponential.
Edit: reviewed the Terraent slides they indeed plan to literally drive a mile long train vertically into the ground. In which case the quadratic argument stands.
At the cost of significant added complexity. It comes down to drilling cost vs installation/ maintenance cost.
The key thing here is that adding 10x height gives you 10x more PE per weight without extra digging costs (or tower costs assuming the cost per unit of height is constant)
Here's how to think of it: by digging one deep hole instead of many shallow holes (with the same amount of digging) you get (n^2 / 2) instead of (n/2) Potential energy. The average height of all the modules is half the height of the shaft. With one deep shaft, the modules pass through the same excavated volume so they can all go deeper on average.