> The only product of spending the energy is a proof that the energy was spent
yes, and if that proof has value - then it's not a waste!
> there is no relationship between the absolute energy spent and the utility of the network
of course there is - security of the blockchain is proportional top energy spent mining it.
> higher the difficulty the more energy wasted in order to maintain the status quo
you got this backwards, difficulty goes higher because more energy is spent (or there is breakthrough in mining efficiency like cpu->gpu, gpu->fpga, fpga->asic, see next paragraph). and miners spend more energy because market tells them it values the security / utility / w/e of the bitcoin more.
> introduction of more efficient hardware into the network increases overall energy consumption because everyone else without cutting edge ASIC technology has to increase their energy output to compensate for the increase in hash rate
this makes no sense.. introduction of more efficient hardware increases the difficulty, not total energy spent. inefficient miner will remain inefficient and not-profitable no matter how much energy they will be spending.