1. Digital Ocean offer small VPS for $5 per month. That's a 50% saving right away!
2. Stick with Ubuntu in the beginning. It's not the best, but it's 100% good enough and has so much support and tutorials out there.
3. If you have a small VPS with not much RAM, definitely set up a swapfile. It gives you virtual RAM for doing RAM heavy things on a small VPS.
4. Use the virtual firewalls offered by your host rather than the server firewalls in the beginning. If you mess up a server firewall you may have to get your host to reset it for you. If you mess up a virtual firewall you can amend it through a web UI and get back to doing things quickly.
5. Learn to read man pages and log files. Between the two you can figure out how to do stuff, and then figure out why it isn't working correctly.
6. In terms of security, use a recent distro, use a firewall close everything you don't need, use SSH keys, and set up secure passwords for everything else, and you will avoid a lot of problems.
7. Keep an eye on resources, programs like top, uptime, free, df, and du will allow you to see what's using up CPU, RAM, or disk space.
8. Learn a relational database. MySQL or Postgres are good choices. This skill will keep you employed for years, almost every business uses a relational database in one way or another!
9. Have fun :)