1: Start lifting heavy weights, focusing on primary lifts, especially Deadlifts and Squats. Start out with no weight on the bar, and add 1 or 2 KG, 3x per week. I used the Stronglifts 5x5 program as a guide. The Deadlifts seem to be key for me, as they exercise my lower back. I also do a lot of situps to increase core strength.
2: Stop sitting for long periods. I use a pomodoro system to break my work up into 25 minute shifts. Of course sometimes I'm deep into something and work for longer, but I always get up and go for a walk after a work sessions of 25 mins or more. This doesn't actually affect my productivity as much as I thought since I'm usually thinking about work stuff while I walk anyway. If anything I am better for the short breaks and thinking time.
Doing those 2 things solved my back issues, hopefully they help you as well.