Unnoticed for years, malware turned Linux and BSD servers into spamming machines | Hacker News Reader

Unnoticed for years, malware turned Linux and BSD servers into spamming machines

net-security.org

133 pointswglb35 comments

You can prevent this kind of thing, or at least make it easier to keep track of.

Add these rules to your iptables firewall:

    -A OUTPUT -m owner --uid-owner Debian-exim -p tcp --dport 25 -j ACCEPT
    -A OUTPUT -p tcp -d 127.0.0.1 --dport 25 -j ACCEPT
    -A OUTPUT -p tcp --dport 25 -m state --state RELATED,ESTABLISHED -j ACCEPT
    -A OUTPUT -p tcp --dport 25 -j LOG --log-level debug --log-uid --log-prefix "smtp_block "
    -A OUTPUT -p tcp --dport 25 -j REJECT

This blocks anyone except exim from sending email. (Note these rules are not tested with IPv6.)

Then, keep a log of every outgoing email, by adding in /etc/exim/exim4.conf.template:

After:

    begin routers

Add:

    traffic_tap:
      unseen
      no_expn
      no_verify
      transport = local_copy
      driver = accept

After:

    begin transports

Add:

    local_copy:
      driver = appendfile
      delivery_date_add
      envelope_to_add
      return_path_add
      maildir_format = true
      create_directory = true
      directory = /var/log/mail_archive/$tod_logfile/

Create /var/log/mail_archive with write access for Debian-exim

Obviously these rules are for exim on Debian, but you can use the ideas for other systems.

Not that there is anything particularly wrong with this, but I find it silly that one should make such customization on their box just to prevent _one_ potential malware hazard. There are hundred other ways that an infected machine can be abused, so IMHO prevention should be done based on attack vectors - that is stopping the machine from getting infected in the first place.

>Not that there is anything particularly wrong with this, but I find it silly that one should make such customization on their box just to prevent _one_ potential malware hazard.

I generally drop all inbound and outbound by default and open the ports in the direction (out vs in) I need on the box. Why not take it one step farther and tie each opening rule to the process/user that actually needs it? That seems like an awesome idea.

If this is the only box that you will ever maintain, then sure, knock yourself out. The problem shows up once you start maintaining hundreds of boxes, because you have an overly complex security setup that does not bring any substantial benefit to the table. It becomes just another thing you have to think about every time you redesign your environment, install new tools, etc.

You are essentially sacrificing simplicity for a false sense of security.

It's a another example of enumerating badness:

http://www.ranum.com/security/computer_security/editorials/d...

which is prevented by only permitting traffic that is supposed to be on your network / going through your NIC.

It's not to prevent just one malware. Virtually all malware (on servers) these days has just one goal: send spam. Plus outbound spam has a very visible and detrimental impact on your reputation so warrants extra defense.

Spam AND DDOS.

Is mining Bitcoin on owned boxes not a thing anymore? I guess because the difficulty has risen too high for ordinary CPUs?

Why is the headline "Linux and BSD" servers when the exploit here is some weird mail software called "DirectMailer" and Joomla/Wordpress ?

There's never been any confusion about the risks of installing big monster bloatware PHP systems on your servers - your eventual owning is a "when, not if" proposition.

A unix server running "DirectMailer" is like a desktop system running the Ask! Toolbar. Pure cluelessness.

I find these types of headlines very clever: it's click-bait, for sure, but you and I clicked it because we honestly thought there was some sort of nasty exploit out there.

Linux is not safe from exploits or hacks but it wants you to be a responsible adult. You need to update your software frequently, make sure you know what you are installing and what you are doing.

Ultimately, I think we need to move away from the marketing gimmicks that cry "ZOMG Z has no viruses and Y has so many and if you get anything it's your OS' fault" to a "be responsible and reasonable. Your OS is a tool".

Well, it may be misleading. But how often do we blame windows for misbehaving programs? Flash, Java, Firefox etc exploits.

Sure, Microsoft deserves some blame for actual exploits in the OS, and some of its products. But, let's be honest here: All of the above is eventually clumped under "the windows ecosystem" and Microsoft/Windows is tacitly blamed for it.

Unnoticed? The DarkMailer/YellSOFT DirectMailer Spamware is known for years. See this post from 2009 how to fight it:

https://www.atomicorp.com/forum/viewtopic.php?f=3&t=3352

But thanks for the nice analysis.

I wouldn't say unnoticed. Security researchers have known about DM for years. It's one of the most annoying pieces of malware to convince people they have because they don't think Linux can be infected.

"We do not know if the paid-for version of DirectMailer also include the backdoor or not."

I wonder if the paid-for version actually exists or if the whole site is just a setup for spreading the malware, by getting people to download the supposedly cracked version. I can't imagine a lot of folks are spending $240 to download spam software from some shady website.

Exploit kits, buying access to botnets, ... are a thing, so I wouldn't be surprised if it is real.

https://www.reddit.com/r/technology/comments/34m5d5/unnotice...

top comment:

> If it was unnoticed for years then the sysadmins were shit. Why is this newsworthy?

> Any sysadmin worth their salt will be monitoring the systems they manage. This thing made outgoing connections every 15 minutes using a cron job. Any 15 year old can create "malware" like this. If you do not notice this for years, you shouldn't be allowed to manage servers.

Not to say there aren't a lot of shitty admins, however -

> compromised via Joomla and Wordpress exploits

Lots of Wordpress installations are not run by someone who should know better.

Hell a lot of servers on the internet are run by people who shouldn't manage servers, not because they're lazy, but because they're aren't admins. Many tools have been created to let people without knowing how to do things do them but the security of those tools has not kept up.

One of my ops folks characterized Wordpress as a rootkit with a blogging package ride along. So hard not to see the ultimate truth in that

That observation is probably related to http://www.bash.org/?949214 :)

What would your monitoring strategy of choice be to notice these things, for a small "hobby" server?

I've apprenticed as a junior sysadmin in the past, and I can easily see how a professional operation would detect or prevent this by design. But a lot of those methods are more than a little over the top for a one-of, single-user, low-end VPS.

I agree that this is a problem and I've been researching this for some time now.

Ideally you would want a HIDS system for small deployments (i.e. a single server) but there doesn't seem to exist too many of those and there's little information about them online (as far as I can find). OSSEC is one example but I'm not convinced it installs too cleanly. At least FreeBSD has a package for it however. Actual configuration would be a mystery.

I think information about this sort of stuff (intrusion detection) seems to be hard to come by in general. At least stuff that is written by knowledgeable people. I haven't seen much about it from major vendors either - it seems to be a black box kind of technology.

I'd love to have a "lightweight" simple daemon that does this. I guess ideally you would monitor logs for stuff, user and group modifications, cron additions etc.

I tried using AIDE for a while, but it was a mild PITA and I was never convinced an attacker with a brain couldn't just patch the database to hide his changes.

tripwire, rkhunter, ... are relatively simple options.

And then hope that your configuration prevents the attacker from getting root or if they do, then they/their automated scripts are not prepared to break them.

RE: hashing files and whatnot:

FreeBSD with a high enough securelevel allows you to set files append-only or immutable so that not even root can touch them.

But then updating the index becomes problematic, you would need to create delta files (which themselves are also immutable) that contain the changed information and combine the data.

Even then, I'm not convinced you could get that much valuable information from a full filesystem hash (or whatever) scan. The amount of information would probably be overwhelming. You would need to apply heuristics or filters. Checking filetypes with libmagic would work, mostly. You could only alert on new executables or whatever, etc..

OSSEC requires you to understand how it works. That's good, but means that you have to devote 10 or 15 hours reading docs and testing before being productive with it.

I recommend it though

Simply logging into your server once a day and typing "top" is a good start. You can usually spot suspicious processes associated with these spamkits. Also, check /var/log/messages, and check root's email.

Also if you run anything like phpbb or wordpress, make sure you keep it updated.

I also like to disable Apache's suexec (as I once had a rootkit that exploited it, and I don't require it for my systems), limit inbound connections with iptables, and limit ssh access to certain ip addresses. Also, get rid of or disable access to any services you don't really need (like ftp, bind, etc).

It's also very important to check all your php/perl scripts for file/sql injection vulnerabilities, as that is one of the most common methods of infiltration.

I'd like to add that it's a good idea to routinely check /var/log/auth.log for successful SSH logins. If you find that onerous because of the volume of breakin attempts, that's a good sign you need something like fail2ban.

Fail2ban isn't foolproof, incidentally. I once nearly caught a worm despite a limit of 6 login attempts per IP, because I had created an account with username and password both "test" that I had since forgotten about, well before I exposed that particular machine to the internet. Oops! An automated script managed to guess that in well under 6 attempts! Fortunately I was physically present at the time and noticed the abnormal load within a few minutes, and killed the offending process (doubtless a local password cracker for privilege escalation) before it could do any damage. Lesson learned there, and it also taught me the value of knowing your machine well enough to sniff when it's not behaving right. On a Windows machine I would have written off the abnormal load, as abnormal load is normal on Windows.

I'd like to add that in my experience simply moving SSH to a port other than 22 reduces the volume of breaking attempts low enough that it's viable to see things in the log.

Adding to this, I've found `lsof -i` more useful for spotting suspicious activity - it shows processes listening on ports as well as established connections. Enabling and checking cronjob logs is also something I wasn't doing before.

My takeaway from this: `Set-Cookie` and `User-Agent` headers are great places to hide stuff and sneak it by IDS.

Quickly check crontabs for all users on your systems:

cat /etc/passwd | cut -f1 -d":" | while read line ; do sudo crontab -u $line -l ; done

Or more simply:

    sudo grep -r ^ /var/spool/cron

(users' crontab are just /var/spool/cron/$username)

Wow, brilliant. Complete speculation, of course, but it is almost as if the owners of DirectMailer set out to troll for the trivially corruptible, and then play them.

This paper states "BSD" all over the place and only in a small code snippet reveals it's FreeBSD specifically. I understand this type of simplification for the purpose of the title but publication itself, which (I'd assume) is targeted at tech savvy people, should be pretty open about the BSD family affected.

Relies on DNS.

Reply on news.ycombinator.com