Paying for the gas itself would not be an externality. Externalities are for example the worldwide damages caused by extreme weather which is caused by climate change, health problems caused by air pollution or the usage of clean water for cooling
Fueling the power plant is an externality for the people building the power plant. You could argue that it increases their costs, but these things are monopolies, with prices set by bought-off politicians. The plant + fuel costs much more than renewables (so ratepayers get screwed), but I'll wager the plant without fuel is still a bit less than solar or wind construction.
It's an argument that's always reeked of whale shit to me. If they really cared about marine sound pollution they'd go after super yachts first.