Use a flavour of Linux. I recommend Debian. Others will recommend CentOS. There are other options, but probably stick with one of these two.
What are you using for your development environment? It might be good to have a local VM of the same distro, even if you're not ready to switch to linux on the desktop. Otherwise you might push stuff to live which break on linux, even if they work on, say, Windows or Mac. I'm thinking of stuff like whether filenames are case sensitive, and the default encoding of files read into Unicode objects (Python can behave differently on Windows and Linux).
Of course, this is all assuming you're using something cross-platform, and not a Windows-specific dev stack.
CentOS is a remix of Redhat's server, so support is generally secondhand and also a lot of the community are pretty sysadmin savvy, so you may get lost in the terse explanations and lack of hand holding.
On Ubuntu you get a wider audience, a lot of "how to" guides as well as a lot of "what do I do now" discussion forums to glean from. As to the flavor of ubuntu, you will only be doing a few things in the GUI (copy files, maybe a backup program, etc.) most of the meat is accessing and editing the config files via a terminal. I Would go with xubuntu as the GUI is not as resource intensive as the others.
If you really want to get down to the hardware, consider CoreOS.