There's also the externality of paying for the natural gas, which is passed on to the consumer in the form of higher energy bills.
It probably also makes sense to include the $800M we are burning per day right now on patriot missiles (assuming the stockpile hasn't been depleted yet).
Paying for the gas itself would not be an externality. Externalities are for example the worldwide damages caused by extreme weather which is caused by climate change, health problems caused by air pollution or the usage of clean water for cooling
Fueling the power plant is an externality for the people building the power plant. You could argue that it increases their costs, but these things are monopolies, with prices set by bought-off politicians. The plant + fuel costs much more than renewables (so ratepayers get screwed), but I'll wager the plant without fuel is still a bit less than solar or wind construction.
It's an argument that's always reeked of whale shit to me. If they really cared about marine sound pollution they'd go after super yachts first.