An idle thought I had related to this is that the real trick to scaling up fusion power may be to
not try and extract the energy "through the walls", as you said.
You know how they tell you to "visualise success" because you need to know what it looks like to aim for it?
Along those lines I was visualising what it would look like if someone like Elon Musk was on YouTube showing off some sort of future fusion reactor that's "not just a piece of lab equipment", but something that would be dramatically better than any extant fission reactor or similar technology.
I visualised it as an enormous stationary rocket engine, where cooling water was pumped through a relatively narrow (~50cm) "reactor tube". The water needs leave a hole in the middle for the fusion fuel gases. The hole is achieved by imparting a rotation to the water, so it "spins out" to the sides. Fusion would occur not throughout a large volume, but at a small number of "pinch points" along the axis set up using powerful magnets -- and not necessarily superconducting magnets! To get sufficient current through the coils, the charged exhaust is allowed to expand through a magnetic "rocket engine bell", producing moving (and accelerating) electric charges. This current is recirculated through the engine to provide the enormous magnetic field strengths required, via thick copper conductors aggressively cooled with water.
Essentially it would be a magneto-hydrodynamic-fusion jet engine, using fields instead of impellers to achieve compression, expansion, and energy recovery to run the whole cycle.
The bulk of the energy would be in the super-heated steam produced, which could be used as in traditional power plants, or used as a bone-fide rocket in space.
Obviously there would be "challenges" to making this work, to put it mildly!
But consider that with the kind of computer power we can throw at modelling physics these days, it may be possible to use "topology optimisation" style tricks to figure out the required geometry of the parts to achieve the conditions required for fusion while staying within material design constraints.
If you ask me, a Manhattan-project style attempt at something like this would be a better way to "waste money" than yet another aircraft carrier, or whatever...