In the same boat here, spend a lot of time in front of the computer and started getting lower back + shoulder + neck aches.

I now do an hour of yoga every morning, which is basically a mixture of breathing exercises and stretching. Keep it simple, look up a couple of basic youtube videos or better still hire someone to teach you the basics. It takes a while to build up the will to do it every morning, but once you've got the basics down your back will thank you.

I use to go swim every other day, but that's been on hold these past couple of months. best of luck.