1. We've raised CO2 from 280ppm to 420ppm, about a 50% increase. To dilute it back down would require 50% more total atmosphere. This would also raise the surface air pressure 1.5x.
2. How much heat is trapped is related to the absolute amount of CO2 in the atmosphere, not the fraction. So the diluted atmosphere would retain just as much heat.
If the latter, it might actually work. Assuming they offgas at-proportion. Which they probably wouldn’t…
Do you really think it's both feasible and a good idea to release so much O2 and N2 to double the mass of the atmosphere? Or even just increase it by some appreciable fraction?
For the record, the atmosphere is around 5 150 000 000 000 000 metric tons. 5 quintillion kilograms. You're talking about producing metric exatons of gas.
Wikipedia says that there's 300 000 to a million gigatons of nitrogen in the earth's crust; that's 300 teratons to a petaton (https://en.wikipedia.org/wiki/Nitrogen#Occurrence). If you extracted LITERALLY ALL THE NITROGEN IN THE CRUST, converted it to nitrogen gas and released it into the atmosphere, and we use the extremely optimistic 1 petaton estimate, you'd have increased the mass of the atmosphere by roughly 1/5000. That means you'd have decreased the CO2 concentration in the atmosphere ... by roughly 1/5000. From 424 ppm to 423.92 ppm.
But guess what, all of those chemicals are extremely valuable, such as nitrates for fertiliser, water, and Argon does really react with anything (it’s a noble gas), which is why we use it as a shield gas in processes like welding.
So producing enough of those gases to somehow offset CO2 production would first require ludicrously large amounts of energy, and if we had access to that amount of clean energy we wouldn’t even be having this discussion. Plus it requires breaking down really valuable chemicals that we spend quite a lot of energy trying to produce or preserve anyway.