The vast bulk of the atmosphere is transparent to infrared radiation, it has no insulation properties at all to speak of.
The only reason our planet isn't a freezing ball is the actual insulation we do have .. and that's the 0.04% you're so flippantly and ignorantly dismissive of.
When that insulation doubles or halves it's a big deal indeed .. and yet still a tiny percentage of the atmosphere.
Try swallowing 0.04% by your body weight of a toxin and tell us how that goes .. by your "logic" that should be trivial to shrug off.
The effect of carbon dioxide on global warming has been proven a million times over. Small percentages can significantly affect heat capture from the sun. A quick overview[1].
Even at 0.1% CO2, you'll feel it (light-headedness, dizzy). Around 3-5% you'll be dead.
CO2 is rising at a rate 100x faster than pretty much ever. But you're right, its only 0.04% i'm sure we'll be fine [2]
[1] https://news.climate.columbia.edu/2019/07/30/co2-drives-glob... [2] https://www.cbsnews.com/news/carbon-dioxide-levels-surging-f...