The nuclear fuel is contained inside a reactor vessel. The water pressure just allows it to be much cheaper.
This sounds like the worst idea I've ever heard.
Fill with unshielded nuclear reactor of novel type: super skinny.
Gently lower down until depth of 1 mile is reached.
Repeat 1000x for a 1 GWe power plant.
What could possibly go wrong? Best horror story of the year in 15 slides.
Or are you also burying the turbine and power lines a mile deep?