Broadly speaking, for lower back health, you should look at anterior chain work, e.g. deadlifts or kettlebell swings. For upper back health, some type of overhead pressing (military press, side press, etc.) I would avoid bench press, as it could make forward head posture worse. In general, I am a fan of Kettlebell work, though any modality can accomplish these things.

I would also add some light mobility/stability work, like the reset video another commenter posted here. I like Original Strength a lot, in particular quadruped rocking and bird dogs.

These in combination should massively improve quality of life for desk guys. A close examination of ergonomics is in order, as well. The challenge is finding programming that works for you, but in general software development is pretty flexible in allowing that.