I dont recall the exact number, but I think that altitude is 50% of the energy, the other 50% is horizontal velocity. So building a rocket on top of the Everest would save us maybe 5% (10 km vs 100 km altitude), that's not much
An orbit at 200 km altitude has an orbital velocity of 7.79 km/s. Potential energy is given by m * g * h (to good approximation, as h is much smaller than the Earth radius of 6300 km) and kinetic energy goes as 0.5 m v^2. Per unit mass (that appears identically in both energy forms) we have potential energy of g * h = 9.81 m / s^2 * 2e5 m = 1.96e6 m^2/s^2. For kinetic energy we find 0.5 * (7.79e3 m/s)^2 = 3e7 m^2/s^2. So only 6% of energy are potential energy, 94% are kinetic energy.
For a relatively high orbit with 1500 km and 7.12 km/s the ratio becomes a more even 37% to 63%. If we include the extra 1.5km/s of delta-v that we loose to drag it becomes 28% to 72%.
At the typical parameters of stage separation from the first stage the split is 3% to 97%. This is also why replacing the first stage by an airplane (that only gets you altitude, not that much speed) does buy you a lot less than you might think at first. We still need a rocket to go to space, even if you start 12km up.
What would happen if we were able to make it all the way up to space, vertically, but not gain any horizontal speed?
Except that it would be better than that, because rocket nozzles work best at a specific external pressure, and getting to launch your rocket in vacuum means you can design your engines for strictly that.
There are occasionally some ideas of launching rockets from tops of tall mountains, like Kilimanjaro. The advantage directly gained from being 6km closer to space is negligible, but the advantage gained from being able to use more expanded nozzles would be substantial -- albeit likely not worth having to haul your rocket up a mountain to launch it.
"Low earth orbit is halfway to anywhere in the solar system."