It doesn't matter if I have 1 GW of solar and batteries and EVs and the whole grid is converted to renewables and I only eat vegan from my own back yard, etc. If my home furnace runs on gas it's emitting CO2 to run. There was literally an exhaust pipe running out the side of my house for the furnace, water heater, etc. The only way to eliminate those emissions is to eliminate the appliance.
I don't know what the break even cost on all this stuff is, because I frankly don't care. I did do the math on the heat pump water heater vs. what we were paying for the gas water heater (there was a time where the water heater was the only gas thing running), and the new water heater saved us about $30/mo in energy costs just comparing gas to electric. And with solar panels the savings over a gas heater are even greater. I figure break even on a very high end water heater is <4 years.
[edits because I mixed up answers to another comment]
Switching to zero-emissions alternatives now means less adjustment later when the polluting versions are (hopefully inevitably) banned.
I don't think this follows. There's not THAT much that goes into making a furnace, but it emits a hell of a lot of emissions every year that it's running. I suspect they paid back the CO2 cost of the new equipment relatively quickly.