The SCD40 series is a pain to calibrate - it will drift out of calibration unless it's left outside at some points. It's not ideal for consumer electronics.
It's just the way CO2 sensing tech works.
I wonder why they don't just calibrate them off the factory line then? I mean, 50ppm is not much. In auto mode you won't get much better accuracy than that because you don't always air the house well enough to get down to 400 (let alone in a city's CO2 bubble).