Kids, this is story of How I Met... my VPS hacked
corrspt.com
corrspt.com
However I still praise him for not only taking time to investigate how the breach happened, but also to blog about it so we can all benefit from the experience.
> And you're right, I don't know and it's probable that they left something else. I'll probably need to nuke the server and rebuild, but I wanted to have a better idea of how I could improve my situation in the future. And I believe that exposing what I did and how I did, the community would be of a assistance.
http://www.reddit.com/r/programming/comments/1vo7zv/kids_thi...
It is well-known for decades (at least since PHP got popular, which was in '90s!) that such locations should only be writeable by the user(s) who maintain the code and not anyone else. Sometimes, such setup can't be done on dirt-cheap FTP-only shared hosting services (shrugs), but certainly not on VPSes.
Ignorance is bliss.
edit: I know a lot of HN'ers are fans of Ubuntu but, FWIW, SELinux (default on RHEL and derivatives) would have prevented this.
I mean, all writeable directories (if any) must not only be not runnable by some server (like Apache with mod_php), but also be on devices mounted with noexec, like this is the case with /tmp.
I can't verify it at the moment, but create a /tmp/phpinfo.php file with "<?php phpinfo(); ?>" in it and then run "php /tmp/phpinfo.php". Regardless of whether /tmp is mounted noexec I think you'll find that it executes (because the php binary is under /usr which almost certainly isn't mounted noexec).
Anyway, it's also better to have noexec than not. :)
At least, it'll help if there's no complete shell access but only ability to run some binary by name, without args, or modify environment variables when running some executable.
Why add more moving parts when they don't do anything but make more work? Scanning is a helpful idea, but not AV scanning. Regular vulnerability scanning can assess the platform security. At the very least, it can warn about potential security holes. It might also be plagued with false positives, causing more work for no added benefit. Safely running services on the internet is hard.
Because in this case we're talking about intentionally accepting files from users to either integrate into the system or offer to other users. Why would you not at least check files for cleanness and reject any that fail instead of blindly accepting them because you wanted to enable file uploads.
Vulnerability scanning isn't going to tell you much when you want to accept files from users.
* (That, and custom programs that use perversely wrong paths, e.g. web content in /home, logs in the application directory, etc.)
If you must, start off with setting SELinux to "Permissive" instead of disabling it completely. Then after a few days of running, go through your audit logs, fix any of the errors that come up, then set it back to Enforcing.
This is the best bet for something that's in production already, but ideally, you want to have SELinux set to enforcing in testing environments and create the policies there in the first place.
The issues you mentioned is very basic that should be avoided in the first place. VPS should be well maintained by the owner himself or have professional service to help.
For a better architecture, a web server should be separated from the app server and database server, because the HTTP port is always open on the firewall, while the rest can only be accessed locally.
I have a blog page showing step by step how to configure application development environment on Windows and deployment environment on Linux. Maybe it's helpful.
http://bingobo.info/blog/contents/how-to-configure-applicati...
Like many CMSes seem intent on doing?
Guess there are always exceptional cases to anything.
Now I'm slightly disappointed. Am I the only one? :\
Both mistakes are amazingly easy to fall into, unfortunately.
Using software released in 2004 without patching up vulnerabilities is probably a more significant problem.
I was ready to give up on JavaEE around version 1.4, but it's so much fun to program JavaEE now. A lot more like TurboGears or RoR (lots of meta-programming and default "scaffolding").
I mean, this is a classic example of badly configured filesystem permissions.
You deploy Play apps inside an application container (like JBoss, Tomcat, Jetty, etc...)
Vert.x is based on Netty and thus it is not running on an application container.
I'd say that Vert.x or Netty based apps are more safer - from this point of view, but there may be some other kind of vulnerabilities for them..
Also, it's important to note that the owner of the VPS instance didn't took proper precautions: "I overlooked the deployment of the web console and HTTP Invoker and I paid for that". If you go by the book and do everything right JBoss and other application containers are safe to use.
I don't like changing my default SSH port, but I don't like people trying to brute-force my SSH passwords either. Instead I use iptables to drop SSH connections from any IP address that attempts to connect overly frequently. This is highly efficient (compared to scripts like fail2ban) and very simple to implement:
# SSH daemon - tcp Port 22 - drop any more than 3 new connections from one address every 5 mins
$IPTABLES -I INPUT -p tcp -i eth+ --dport 22 -m state --state NEW -m recent --set
$IPTABLES -I INPUT -p tcp -i eth+ --dport 22 -m state --state NEW -m recent --update --seconds 300 --hitcount 3 -j DROP
$IPTABLES -A INPUT -p tcp -i eth+ --dport 22 -j ACCEPT
For my use case though, this reduces load on my server (and prevents clogging my auth log files) by stopping incessant password brute-forcing attempts. I must admit to quickly adding an over-riding ALLOW for the handful of IP addresses that should have access, though!
In the long run, I've found that years pass and connections configs get lost and you forget which fancy port you used for your SSH connection on that server. Maybe YOU have an ironclad convention, but your co-worker had another one, and you can't remember what port he used. And he's left the company or died or joined a cult.
Kids, leave your SSH ports alone. A config is just a config. But keys are forever.
That's naturally not the only layer of security, but I figure it's a nicer option than non-default port.
The Chinese attacks use a group of about 15 IP addresses, then, every so often, they all change the addresses to new ones at once. This has just happened, last week, in fact. So now I have dozens of attackers all coming from a group of about 15 IP addresses, which are different to the 15 or so IP addresses they used a couple of weeks ago. (No kidding, the regularity that this happens, it would not surprise me if their military is training a new class of crackers and has been assigned a different set of addresses to use this term.)
When I get a new IP address in the log, I do a whois and rewrite the "inetnum:|NetRange:" field to a class A|B|C address and then DROP it in iptables. Fuck 'em. The whole darn network class gets dropped. Not that I'm likely to be logging in from China any time soon anyway.
I now have a list of network classes with about 35 address ranges that get dropped, if anyone is interested in the list.
Binding to IPv6-only is more effective at reducing log spam: IP scanning 2^128 addresses is impractical, and scanners often cannot connect because of misconfiguration/incompatibility or lack of a routable IPv6 host address.
Isn't this likely to keep you out of your own system too, at some point (accessing from unusual location without IPv6)?
So sure, change the port and all, just as long as you're aware of what it does and doesn't do. I don't think anyone believes that you can allow root login with a password of 12345 if only you change the port, but it can be a good layer in a well-designed system.
If your box is more vulnerable on port 22 than port 22221 then your problems run orders of magnitude deeper than which port your ssh server runs on.
A rather horrible practice to begin with. There's always so much that can go wrong when parsing tainted strings to native data structures, let alone to full objects using the builtin marshalling functions, they're not meant to be used on objects from untrusted sources at all!
Of course, the correct answer is not to use the deserializer that can instantiate arbitrary classes when you have a well-defined list of classes that can be instantiated.
Original URL is http://pdd-nos.info/.tmp/back.conn.txt
It's one the oldest linux distros and doesn't really hold your hand that much. You'll be editing a lot of config files by hand. However once you get used to it you'll definitely feel a much stronger connection to your box than with a lot of distros that try to make things easier. /r/slackware is a good resource, there are a good amount of lurkers that jump at any chance to help someone out, and linuxquestions.org is a good forum as well.
EDIT: forgot to mention: don't rule out FreeBSD/OpenBSD as well. They are known to be pretty solid for hosting as well.
Here's my advice for a secure public-facing server: use Debian Stable, set up automatic upgrades every night, install as much as possible from the official Debian repository, and be sure to upgrade to the next Debian release before your current release loses security support. This way, the Debian Security Team is responsible for monitoring security advisories and rebuilding packages instead of you having to do it yourself.
Source: I just finished transitioning to Debian after 10 years of Slackware use, in large part because I found it too difficult to keep my Slackware installations secure.
Avoid consumer-focused distros like Ubuntu or Fedora; you'll also learn more by avoiding them as they tend to hand-hold the user and hide functionality and configuration from them.
NEVER GIVE THE USER YOUR APPLICATION SERVER RUNS UNDER SUDO PERMISSIONS!
Reminds me of wordpress attacks, you quickly wish for FS diffs in order to identify any change in your code/data ...
It's pretty much a set it and forget it mining.