Remove too much carbon however and the plants die.
We don't want a Great Decarbonification Event.
Decaying plants emit carbon. The extra nitrogen would just be changing is the length of the give-your-carbon-back-to-the-atmosphere queue. It's negative feedback: stable.