If I could give one piece of advice to every startup who is worried about server security, it would be this: read the NSA's two page pamphlet for hardening Red Hat Enterprise Linux [1] for a quick overview and then move on to the ~200 page complete guide [2]. The principles are generally applicable to most Linux systems.
The NSA's guide is good enough for low value targets. Besides, these are your tax dollars at work, so you really should make use of them. :-p
[1] http://www.nsa.gov/ia/_files/factsheets/rhel5-pamphlet-i731....
[2] http://www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf
On the web application side, you're going to want OWASP. http://www.owasp.org/index.php/Main_Page
EDIT: Just thought of something.... There were several privilege escalation exploits for the kernel made public around this time frame [3]. I'm guessing that's the reason why his kernel was silently upgraded. :-p