Hacker News Reader

Comment by tyingq | Hacker News Reader

Looks like it was old school shared web hosting with things like WHMCS, Cpanel, Softaculous, Wordpress one-click installers, in-house written web admin, etc. Where each user was running in the same instance of Linux. Not, for example, a VM per user. That kind of setup has a really wide attack surface. Not surprising it was hacked. That kind of setup with deliberately narrow support was bound to get hit.

Edit: Apparently their "sister company" sells virtual private servers, and appears to still be alive. http://nosupportvpshosting.com/index.php

View on HN

Most shared hostings work like that, they do not have a VM per user, and the professional ones don't get hacked often.

I used to manage a fleet of cPanel servers years back, they were a pain to manage as users would regularly get infected on their local machines and have HTML infected with malicious javascript. We used a bunch of tools like clamav/configserver etc to keep on top of it but it was definitely whack-a-mole. Even with inotify style job triggers to check the newly uploaded content.

I set up a virtual guestbook for my daughter's birthday party last year on a cPanel-based hosting system. Couldn't for the life of me figure out why the damned thing wasn't working until I discovered that the admin had gone ahead and disabled my link "guestbook.cgi" since it was a common cPanel attack point. Apparently cPanel comes with a hackable default guest book of its own.

How do they secure php? The php process can access nearly anything. Also they may have remote code execution vuln: https://www.trendmicro.com/vinfo/us/security/news/vulnerabil...

They use php-fpm pools where each website gets it's own uid:gid and the php process runs as that user. Then standard linux file permissions so you can only access your own uid files. To access web assets from nginx/apache, they add file permissions wich standard Linux acls.

Altough not fancy, the security model is actually quite mature. Security problems in these servers come from misconfigured permissions and scripts, not the security stack.

Supporting shared hosting well is probably a large reason for PHP's success.

Sure, but there's nothing inherent to the language that fueled that. They could have done that with Python, for example.

Rather, it was popular software like bulletin boards and blogging platforms that built the demand. PHP used to have one of the lowest barriers to entry because you could get by with plain HTML and incrementally add business logic inline.

> Sure, but there's nothing inherent to the language that fueled that. They could have done that with Python, for example.

Not quite true. With shared hosting it was (is?) uncommon for a user directory to have ExecCGI enabled. If you wanted scripts to run they had to live in the cgi-bin directory. Additionally mod_rewrite could be expensive on low powered servers. This all meant doing anything dynamic meant "ugly" URLs and meta tag forwards if you were on such a shared host. It was also non-trivial amounts of effort to get some random CGI script working since you needed to know enough to get the shebang path correct for the server and set the right permissions.

Contrast this to PHP where you dropped a .php file into your user directory and you've got some dynamic content. Platforms built on PHP became popular because you could upload them to your user folder and they just sort of worked. There were no special executable paths, no shebangs, and no execute permissions to set.

Perl was huge in the CGI space for a long time but the (consumer) content platforms built on it weren't nearly as successful because of the difficulty of mere mortals getting them running on their shared hosting plans.

Not really; PHP had many features to make this easier; for example "safe_mode" and "open_basedir". These are not easily replicated in a stock Python by just "flicking a switch", even today (although the need for that today is a lot less than it was in 2000, and PHP even removed safe_mode). Not that these measures were perfect, but they were mostly "good enough".

There was a reason that in ~2000-2005 you could find PHP shared hosts for $1/$2 month, and that Python/Perl/etc. shared hosts were much harder to find and more expensive. People started using PHP bulletin boards and blogging platforms because at the time it was easier and cheaper to run, but that's an effect and not a cause.

> These are not easily replicated in a stock Python by just "flicking a switch", even today.

Chroot, containers/jails, SElinux, these are all OSwide and would protect PHP just as much a Python or nodejs. There’s a reason PHP is dying off.

These either weren't available at all in 2000 or very new and much harder to set up. You need to consider the state of things 20 years ago, not how they are today.

chroot and jails have been around for a while, i don’t know the date but since ive used bsd and linux they’ve been there. definitely during this era. selinux was launched in 2000.

Sure, you could even argue that PHP was a success despite the PHP language (which was, in the beginning, only a cobbled-together templating language, and then, to Rasmus Lerdorf's dismay, people started to implement their backend logic in the templating language), and Python could have done it, but, well, they didn't...

now, you can even write actors in PHP: https://github.com/dapr/php-sdk#actors

Deployment?

PHP made "deployment" very simple. On the other hand, Python is complicated to deploy even today.

I’ve always found that very interesting about PHP. It can do both, be a standalone scripting language, and a templating engine.

Not that I would ever recommend using as a templating language in 2021 :) but it’s cool that it can do that without any external library.

It's so cool that even though it's already a templating language, it's a templating language that hosts other templating languages too (Smarty, Twig).

PHP succeed precisely because of how it executes .php files directly as scripts from a single running instance. This allowed it to be the shared hosting solution because it uses far far less resources than a similar python solution.

Python is great if you want to run your own server or VM per site. It's not so great if you want to run a shared server and it's shared servers (not VMs) that brought the price down and therefore opened the floodgates of allowing people to run their own sites cheaply and easily.

Python webserver support doesn't support SSI-style open and closing code tags in HTML files to be executed per-request, last I checked.

Is there anything like php or bml (bradfitz's equivalent for perl) for python, so that you can put code right in your html to be replaced at serving time with the code's output?

https://docs.python.org/3/library/cgi.html All your prints are essentially sent as response with CGI, but I think for Python WSGI is the standard you should use, and you can e.g. use jinja to render HTML with templates where you can use variables, certain functions, etc. Is that what you are looking for?

PSP (python server pages) is pretty close to that. I don't think it is terribly popular. There's also Spyce.

With the recent problem in sudo, I suspect this to be a likely cause. The typical shared hosting stack uses that somewhere, so servers will have it installed. A fast, malicious user (and a slow update process) is enough to get root on one machine and penetrate the rest of the net from there (can still be avoided, but requires some effort).

Just a guess, of course.

Unix permissions and suexec I seem to recall, with other mitigations (chroot I think, but it's been a while since used it)

I assume because they have more margin to keep up. If you work a system like this with $1/user minus the 30 cent Cpanel license, you have little to spend on security.

The "no vm per user" means any privilege escalation bug lets a hacker wipe it all. And your unsupported customers are probably running all sorts of vulnerable stuff.

Sounds like a Docker problem. Have the isolation without the VM overhead and associated network topology.

When I saw that customers needed to access cpanel and download their backups, I was wondering how this could be described as a place for experts.

If each of those services were running on different docker instances it wouldn't have happened (potentially)?

CVEs for breaking out of a docker container come along as well [1]. Usually you need root in the docker container, but if you combine it with an escalation from non-root to root.. well, you can see how that's less secure than a VM

[1] https://unit42.paloaltonetworks.com/breaking-docker-via-runc...

Everything can be hacked so if the decision is on what can or cannot be hacked then just unplug the server.

Docker containers are not security boundaries, unless ran on top of firecracker or gVisor.

why not? I would argue, that they use linux namespacing, cgroup etc.

In theory it is a security boundary. But the attack surface is so big, and local privilege escalation bugs so common, that you should not rely on it to isolate different untrusted users.

You can say the same about VMs, to some extent.

Containers absolutely are intended to be a security boundary.

They most certainly are not. That's a common misbelief. Containers are not designed as a security boundary, they just happen to function as one most of the time.

VMs on the other hand actually are designed as a security boundary, but even then there are still attacks you can do against other VMs on the same box.

I agree that both are security boundaries in theory. But a minimal hypervisor is much stronger than a cgroup container. Cgroup containers are a thin door made of wood, VMs a vault door made of steel. So people saying "containers are no security boundary" are exaggerating a bit, but not much.

A minimal VM, like firecracker has a small attack surface, so I'm willing to trust that privilege escalation/VM escapes will be rare.

A process restricted by cgroup/namespace/etc. still has access to the huge API surface exposed by the kernel, so privilege escalation is common, and I'm unwilling to trust this mechanism to isolate malicious code.

I agree that they're not very good ones, but a container escape would be treated by everyone the same way a VM escape would be: instant patching, coordinated/embargoed disclosure, AWS finding out before you do, et c.

They didn't start out at the design phase that way, but they absolutely are today.

https://en.wikipedia.org/wiki/Virtual_machine_escape

Of course VMs escapes exist. But many of the vulnerabilities are in functionality which aren't relevant for modern servers. Hardware virtualization support prevents many attacks. For example firecracker supports little more than network, block-storage and vsocks, which keeps the attack surface small.

Containers are not intended to be a security boundary -- functionality along those lines has been gradually backported as maintainers realized that nobody was going to care when they said "don't use these as a security boundary".

There's a world of difference between the amalgamation of hacks that comprise cgroups and something like BSD jails, which are and afaik always have been intended to be a security boundary, which implements real first-class kernel isolation for jailed processes, not just another subtree under proc that provides some direction to the kernel around resource consumption/priority and relies on UID/GID hacks to control access.

No they are not

https://info.aquasec.com/container-security-book

You expect people to read a book to find out your perspective? Do you have cliff notes on why using isolation mode doesn’t provide a security boundary?

Containers provide resource isolation using a shared kernel but are not intended to be used in hostile multitenancy scenarios.

A key feature of OS virtualisation is the strong segmentation boundary between

1. Guests

2. Guests and the hypervisor.

For this reason, VMs are seen to provide a stronger security boundary than containers and are used in preference where that aspect is critical owing to environment, multi-tenancy, business context.

So again, what about isolation mode? I don’t know what this is called in the linux world but in windows this feature does exactly this. Still a shared kernel but a far cry from what your explaining.

It's not good enough for multi tenant setups. A single malicious customer can potentially steal data from other customers. The docker team also considers security to be a pretty low priority.

To say nothing of sidechannel attacks[0]

People need to stop looking at containers as a cheap way to get security. They might be a more convenient way to get lots of apps running on a single machine, but they're not very secure.

https://ieeexplore.ieee.org/document/7847002

1. I expect people to move towards a VM per pod model, even in private setups. Firecracker claims a memory overhead of 5 MB, and a minimal QEMU setup shouldn't be too bad either.

2. It sounds like this paper is mainly about covert channels not side channels. Covert channels assume cooperation between both sides, so they're only relevant if one of the sides can't communicate trivially (e.g. via network)

> vm per pod .. firecracker

agreed. AWS gets a lot of flak, but open sourcing firecracker was really great. I'd really prefer to see us move toward vms instead of containers, even if we kept the same k8s abstractions.

> .. covert ..

thanks for the catch, should have taken more time. Here's a better paper:

https://hal.inria.fr/hal-01591808/document

> I'd really prefer to see us move toward vms instead of containers, even if we kept the same k8s abstractions

1. For me containers are one of those abstractions, defined by exposing an application controlled userspace. Containers can be implemented by different isolation technologies, from simple chroot/cgroup/namespaces... to VMs.

2. I'd still use chroot&co to partially isolate containers within a pod, while using VMs to strongly isolate pods from each other. This enables features like shared block-devices, unix-domain-sockets and monitoring the processes in an application container from a separate diagnostics container.

I think it's easier to say that namespacing is nearly orthogonal to security. Native containers (i.e. containers not running in a VM) are literally just processes running on the host and need to be secured with the same methods you would use on non-namespaced processes. Namespacing does add another layer when used properly but it doesn't replace any of the existing ones.

I agree that containers are something of a security boundary, as is chroot(). Just not as robust a boundary as an actual VM.

in the olden days, VPS providers worth their salt ran freebsd and had each tenant inside a jail.

This was back in the early 2000's, and still seems to work rather well for the basic webhosting.

Why not use a 'proper' VM system like xen?

Xen only hit v1 in 2004 and was relatively niche.

Lower overhead, both from a system and an administrator perspective.

Yeah but it would be perfect for sysadmin who just want to host python, node type of server. Too bad they are shutting down.

http??

Yeah. There is an https endpoint, but the page renders wrong for me, so I linked the http one. I didn't look to see if the client login/registration sends plaintext over http.

Edit: It does login over http, plaintext passwords over the wire. Heh.

It is also possible they wanted to shut down this service and pretending a hacker caused damage is an easy way out.

Reply on news.ycombinator.com