SELinux is beyond saving | Hacker News Reader

SELinux is beyond saving

utcc.utoronto.ca

137 pointsfcambus77 comments

When CentOS 7 came out, I decided to make peace with all of the new stuff (systemd) and all of the old stuff I had been disabling (SELinux.)

Turns out that doing crazy shit like letting users have their html files in ~/public_html/ requires a lot of SELinux configuration. procmail touching user directories? Yep. spamassassin? Why, yes. Maybe there's something I did wrong... I did read the docs.

Also turns out that there isn't a tool which tells you what new rules are needed, relative to the existing configuration, for recent SELinux denies. Yeah, there are some tools to spit out a complete config file based on all logged problems, but not a diff, and I had already lost some of the early logs to logrotate n=4 by the time I realized I needed 'em.

111 lines of perl and 116 lines of SELinux rules later, I was in good shape. But REALLY? REALLY?

The targeted policies and functionality booleans in CentOS are really nice. I took the same step with finally buckling down and actually using SELinux for version 6 and there was definitely a learning cliff, but just like anything else after you've used it for a while it's all old hat.

You may have seen it already, but the CentOS SELinux HOWTO was very useful for me in learning how to interact with it: https://wiki.centos.org/HowTos/SELinux

>Turns out that doing crazy shit like letting users have their html files in ~/public_html/ requires a lot of SELinux configuration

You should just be able to `setsebool httpd_enable_homedirs on`. What trouble did you run in to?

>procmail touching user directories?

This works by default for me in in C6 and C7: I'm using it on a number of production systems. The ~/.procmailrc should have type procmail_home_t but procmail itself can create/modify/delete files in the user's home dir.

>spamassassin?

Again, should just work. You may need to `setsebool spamassassin_can_network on` depending on configuration.

>a tool which tells you what new rules are needed

That would be nice. My usual process if I'm in a situation that needs custom rules is to:

    echo '' >/var/log/audit/audit.log # (it would be better to rotate it here, but you get the idea)
    setenforce 0
    # Do whatever I want to be able to do...
    setenforce 1
    sealert -a /var/log/audit/audit.log

I read that HOWTO, actually, and the stuff you're describing isn't really explained in it.

For example, there's a httpd example changing a label right there in the HOWTO, and it doesn't explain where I'm supposed to discover the correct label. And the recommended audit-script-to-config doesn't show the correct labels, it prompts you to relax the rule instead of changing labels.

My procmail rules don't correspond to what you said procmail does. And how was I supposed to discover procmail_home_t ?

I'm not saying I'm Mr. Super Smart, I'm just saying that it's quite a bit of investment.

The man pages are useful, especially once you know the basics. For example, start at [man selinux](http://linux.die.net/man/8/selinux) in See Also see the reference to [man httpd_selinux](http://linux.die.net/man/8/httpd_selinux) and in there you find every type and boolean including `httpd_enable_homedirs`. See also, [man procmail_selinux](http://linux.die.net/man/8/procmail_selinux).

> I read that HOWTO, actually, and the stuff you're describing isn't really explained in it.

Did you read this part? It explains a solution to your first gripe: https://wiki.centos.org/HowTos/SELinux#head-0f6390ddacfab39e...

I find audit2why helpful, as well as semanage boolean -l, semanage user -l, semanage fcontext -l. Often times more than not, these tools either help me fix an 'audit' issue or point me in the right direction. Check them out sometime, if you don't want to go digging through docs or serverfault - the labels tend to be generally straight forward IMO :-)

You'll have to add a -P to `setsebool httpd_enable_homedirs on` to be super-effective.

Which sorta supports the OP's point.

The -P means persist through a reboot.

> Turns out that doing crazy shit like letting users have their html files in ~/public_html/ requires a lot of SELinux configuration

I guess "setsebool httpd_enable_homedirs on" would do that.

> Also turns out that there isn't a tool which tells you what new rules are needed, relative to the existing configuration, for recent SELinux denies

setroubleshoot should be shipped with RedHat-based distros. Mentioned in https://youtu.be/cNoVgDqqJmM

Besides that, there's domain-based switch-off command, "semanage permissive --add programname_t"

> 111 lines of perl and 116 lines of SELinux rules later, I was in good shape.

I paid a SELinux developer and just nice guy from #gentoo-hardened, perfinion, to help me with setting up my workstations so that I could run untrusted crap like skype in enforced mode. After that, I was in good shape and full of warm cozy safe feeling. And now I can use SELinux hardening stuff at my job.

The truly scary part for me is that I might have missed one little thing that will come back and bite me. I have two RedHat boxes because a vendor requires them for software[1]. Luckily, neither needs to service internet clients.

1) the same vendor that feels a 752 Mbyte SQL file constitutes a patch

The server I'm talking about is a multi-user box used by a bunch of people for various toy activities. So what usually happens is someone will try to install something new and reasonable, it will fail, they will bitch at me, and then I'll run my scripts and update the selinux config.

However, I do end up with surprising mystery issues. Just got a rule for logrotate needing to be able to reload some daemon... a year into this experiment. Wut?

Thankfully, my servers are in full appliance mode and I don't have to worry about users (they are on the FreeBSD / ZFS box or one of the OpenBSD machines). I still worry since they have pretty valuable data on them.

Yeah, I was seriously ready to destroy furniture when I had to do it for a nonstandard software installation at work, a few years ago. I had to do a crapload of ausearch and audit2allow, and I possibly ended up with a too-permissive rule set. Who knows?

It's worth noting that Android is using SELinux in recent versions for system hardening and as part of the sandboxing model.

See: https://source.android.com/security/selinux/

While I agree usability is suboptimal on servers, there are definitely environments it can excel in.

I don't think android is really a special case, SELinux is still hard. The only thing that makes android special is that google engineers have put in a lot of work to make SELinux work. It's a powerful tool, but if you want the benefits of SELinux it takes significant effort somewhere in the development chain.

I think Android is more of a special-case scenario as Google controls everything and it can just tell developers to conform with the more restricted environment in the new version of Android. For Android it also doesn't matter whether SELinux takes a lot of time to set-up right.

Preventing many 0day or canned exploits isn't exactly suboptimal. It just requires better or more educated sysadmins.

Russell Coker has done some work on SELinux. He's had a Linux box online since 2008 or so that users may ssh into as root. It's really amazing to see what SELinux can do. However, I agree that OpenBSD's focus on small and simple is the better approach.

https://www.coker.com.au/selinux/

https://www.coker.com.au/selinux/play.html

Leaving selinux on would be more easy if the access denials were put in the primary system log and/or in dmesg and did not just go in the audit log. It takes too long to figure out what the problem is with selinux. That being said we leave it on in production, since with ansible we find the problem once and then incorporate the changes into our playbooks.

I've used both apparmor and selinux and while it was pretty easy to figure out how to write a custom profile for apparmor, it's not easy to do the same for selinux. If a custom profile for selinux could be made as easy to write, that would help a lot.

At least on Fedora, for a few releases now, the audit messages appear in the journal, which is the primary system log.

For desktop Linux, SELinux is indeed difficult to use on a day-to-day basis. To the point that there's no way I'd consider using it.

However, it is working great for Android. Where all apps are already sandboxed, and you don't have to worry about the user installing some app that will use a new system call, or try to touch random files on the system.

>For desktop Linux, SELinux is indeed difficult to use on a day-to-day basis. To the point that there's no way I'd consider using it.

It's on by default in Fedora, has been for years. Most people never even know it's there.

Funny, I've been running it on my laptop since Fedora 18 (current on 24 Beta). I might notice it maybe once every few months when a script or some software I'm futzing with tries to do something out of the ordinary:

    [jeff@omniscience ~]$ sestatus
    SELinux status:                 enabled
    SELinuxfs mount:                /sys/fs/selinux
    SELinux root directory:         /etc/selinux
    Loaded policy name:             targeted
    Current mode:                   enforcing
    Mode from config file:          enforcing
    Policy MLS status:              enabled
    Policy deny_unknown status:     allowed
    Max kernel policy version:      30

And yes, my nick is a throwback to Security Enhanced Linux, but when the next exploit for some common library or software comes out and I'm not affected due to SELinux, I'll smile. Can you say the same?

A few quick examples I pulled up:

https://blog.namei.org/2008/01/08/selinux-mitigates-remote-r...

http://danwalsh.livejournal.com/10131.html

https://www.reddit.com/r/linux/comments/1xdokz/selinux_saved...

When SELinux first came out I was working as a training instructor for RH. Every single knowledgable sysadmin was kicked in the teeth by 'avc denied', one of the dumbest error messages to ever be created in Unix.

'avc' was Access Vector Cache. The Access Vector Cache was a component of SELinux. AVC denied meant selinux was denying access. But instead of printing 'selinux denied' - you know, like smb messages are for samba, and postfix messages are for postfix - the SELinux folks seemingly just wanted their audience to do a web search for 'avc denied'.

It's a small picky thing, but rather than one person fix something, they made every admin do a piece of research instead.

This is a pity: SELinux is one of the things that fixes the peering (aka 'Containers don't contain') issue with Docker.

May I disagree partially, using this authors exact words?

"... Unless you are a high risk target, spending almost any time beating SELinux into shape on your machine is a bad tradeoff and pretty much a waste ..." -- https://utcc.utoronto.ca/~cks/space/blog/linux/SELinuxToxicM...

If you are a high risk target, SELinux is a great option.

Also, to refer to the partial disagreement, here is my exact feeling...

"I'm a security aware sysadmin and yet yesterday I casually admitted that I made less-secure choices because the really secure option was too annoying and potentially inconvenient. In fact this is not the only case where I make this tradeoff, picking a less secure but more convenient option..." -- https://utcc.utoronto.ca/~cks/space/blog/tech/SecurityNotImp...

I don't think you're disagreeing with the author. His statement was equivalent to "If you are not a high risk target, SELinux may be a bad choice". Since he used 'unless', I'm assuming his intent was the same as yours, "If you are a high risk target, SELinux is a great option."

I was simply disagreeing with the title of "SELinux is beyond saving." SELinux doesn't need saving for those who have a strong use-case. There are some great lesser alternatives like AppArmor, grsec, TOMOYO, and AKARI if SELinux can't work for you. :)

I did a quick review of available solutions some time ago, and what you're listing (apart from apparmor) is not an alternative. I'm assuming that if you're serious about security and capable of digging into complexity, you can just stay with selinux. So if you lack those:

grsec (rbac) - distributions don't really include it apart from arch, gentoo, and other high-maintenance ones; default targetted configs not included

tomoyo - easy to use for developers, but also often not available in default kernel; for normal users? start by explaining to them what syscalls and ioctls are; default targetted configs not included

akari - you're on your own to compile it in, and the tools, and figure out which version you want, and ... (not simple process); default targetted configs not included

apparmor - the only user-friendly alternative right now

> There are some great lesser alternatives like AppArmor, grsec,

Lesser? grsecurity is actually a far superior alternative to SELinux: https://grsecurity.net/compare.php

It's unfortunate that their stable patches are no longer available publicly: https://grsecurity.net/announce.php

Well, unless you've been blocked from getting it by what's-his-name.

SELinux worked and worked well when you knew how to use it. (Learning how to use it was also reasonably hard.) Its primary problem was that its default behavior was stuff mysteriously failing on your system. This made it very hard for the uninitiated to get into it when stuff all over the web just recommended "turn off SELinux" as the first thing to do when something wasn't working right.

It got a bad reputation as that quirky security thing that keeps stuff from working all the time.

I confess to disabling SELinux being one of the first things I always do on a new Linux box. It gets in the fucking way. A lot.

It works great on really mature platforms where the engineer has time to fine-tune and troubleshoot SELinux issues every time a change is made...

But who the heck has the time for that now-a-days?

I confess to using password "sex" everywhere and pinning it to my workstations' displays. Because making hard and secret passwords is hard.

Configuring SELinux reminded me of configuring email. My god, can simple systems be made more complex than this? I mean it's email and yet you have to really struggle with 100s of options to make sense of it all (and you do know sendmail has no public repo with a proper revision history right?).

Apparmor, in contrast, is much easier to use (granted it has it's warts) but atleast they are trying to be user friendly instead of mathematically complete.

/rant

I remember going through hell trying to build an apparmour profile for an app at my last job (a Python app, which makes this a bit more complicated with interpreters, wrappers, external commands and such).

I did get it built and the built in tools for aa are decent.

I've never had to develop and SELinux profile. Now I don't ever want to.

I feel I need to quality what I meant by 'much easier'. AA is exactly as you made it sound - actively asocial. Now imagine how SELinux is ;)

The cynic in me thinks this is a ploy for Red Hat to be able to sell more RHEL subscriptions/support. It just feels like there is no incentive to make things better.

I'm sure it's not intentional, but the author spends quite a lot of time telling me that the SELinux folks aren't listening or learning what the real issues are in security, but try as I might I all his links to his blog posts he doesn't actually state what they are!

What are the key issues that SELinux needs to address?

One repeated many times is that it's complex enough that people default to turning it off. But it doesn't even come from users sometimes. Just google the following: installation instructions "setenforce 0"

How many projects just went the lazy way? How many projects went out of their way to fix the issue the wrong way?

Notice how nagios documentation first tells you how to disable selinux for the whole system and make it permanent. Only then as an alternative gives you two lines to make it work correctly instead.

So what can SELinux folks address? Submit a change to every documentation in that list with instructions on how to make selinux work without issues. And make sure the alternative of "setenforce 0" has a warning that this will disable protection of your whole system rather than just this app.

Complexity is evil in security, and opt-in security generally doesn't work. SELinux is both complex and opt-in.

You can make it enforce rules strictly over everything, rather than selectively enforce it on specific applications.

no software is written with this in mind, so it wont work right without extensive babysitting (that nobody will do).

SELinux is not opt-in in RedHat-based distros AFAIK.

SELinux is still nice for hardening a safety critical server.

But configuring it sure isn't fun.

The post succeeds at saying nothing concrete about actual deficiencies in SELinux beyond "it gets in the way". Yes of course it does. That's what locked down systems do even if it's only to shoulder the blame for breaches on admins who disabled protections.

My experience with it was on RHEL4.x some years ago, trying to install a bunch of stuff not from the official repos or EPEL. (I think it was Puppet, at the time.) Holy crap. It took me a week. It was more complex (or, almost equivalently, poorly documented) than the old AT&T Merlin PBX+voicemail system.

The only thing I recall offhand was the hugeness of the namespace for roles (?) and actions (?). Every time I punched a hole to allow a specific thing to happen, something else blocked the action with cryptic logs in obscure places. I felt like a blind person, trying to find their way through a maze.

SELinux is not not worth saving because security has been moving steadily into namespace separation (E.g., containers, virtual machines).

Giving an application/user and entire trimmed down OS namespace reduces damage done from getting pwned significantly. Why use a system that protects processes from running alongside each other when I can just give them their own, fenced-in home?

No. No, no, no. Virtual machines - yes, to an extent. Containers - no.

Containers still use the same kernel, have access to the same filesystems (in practice), same devices, same ioctls, etc. There can be a lot more separation done in the future, but right now, you definitely gain a lot by isolating your containers using selinux.

Even with virtual machines, you want any breakout / external write vulnerability to be contained. That's why libvirt/kvm relies on selinux (or apparmor) for system protection and for cross-vm protection (https://libvirt.org/drvqemu.html#securityselinux)

I felt the same way about Docker containers. Now I work for a company that uses Docker containers in production.

The fact that the process, by default, runs as root and PID 1 within the docker container still has be going "You gotta be fucking kidding me?!"

I realize Docker has come a long way since the days of it being LXC based. Cgroups and several other safeguards have done a lot of docker security, and containers are much more lightweight than full VMs, but I'm still a little on edge about the security aspect.

Since Docker 1.10, you can specify --userns-remap=default on the daemon and root inside the container will not be root outside: https://docs.docker.com/engine/reference/commandline/daemon/...

I have the impression that regrettably many docker users expect things to be the other way around: that the actual default docker behaviour resides behind a --ridiculously-insecure, not that you need to specify --i-actually-want-my-container-to-try-to-contain to get that expected behaviour.

Hopefully more people are becoming aware of the actual default behaviour.

Perhaps I'm misunderstanding the second line of your post, so I apologize if that's the case, but the whole idea of the PID namespace is so that the container's init can run as pid 1 in the container without access to the rest of the host's processes. Likewise for root and user namespaces. (Though I understand that one is really tricky to get right)

I realize the system is not perfect, but it's so much better than nothing, and significantly easier to use than virtualmachines/selinux.

It's naive to dismiss something because it doesn't meet the goal of "100%" secure. In practice it gets you very close and takes almost no effort, no maintenance, no headache. Historically that kind of thing will win in the long run.

I'm not arguing that VMs are not an improvement in many areas. They are a MASSIVE improvement and I'd love the unikernel apps to become more popular and push it even further. QubesOS also shows the power of safe virtualisation.

But it doesn't mean "SELinux is not worth saving". Some kind of LSM (selinux/apparmor) is still a great safety net and protects us from an issue that VMs actually introduce. Once you break out of your VM, you have access not to a single host, but to a number of them in a way that no in-VM protection can stop or even detect. That's the reason we really need LSM protection. Not necessarily selinux, but something needs to be there.

How would you run VoIP calls in skype? You would have hard time passing your media sources and outputs through VM or container. Pulseaudio doesn't work fine even if you run two audio activities from two different system users.

Also moving data across VM/container boundary all the time is not a fun. Been there, done that even in lighter manner (with separation of system accounts for different activities).

Sound works quite well in VMs actually. You just get a virtualised audio device which then plays into system's pulseaudio. It "just works" for me. Potential pulseaudio issues are not VM audio issues.

As long as you establish some read-write and read-only shares between vms/containers, there should be no problem with moving the data. As in, you don't have to move it in the first place, but you need to design it the right way.

The article (and the comments here) encouraged me install SELinux on one of my (Debian) VMs to see how it goes, and I'm now wondering why I didn't try it earlier.

If you are a security purist, why aren't you using grsec / pax / rbac in the first place? The general totem pole is grsec rbac is king > Apparmor is a destitute wannabe hacked together disaster that is still more usable than SELinux > Archlinux doesn't give a shit while Red Hat is off doing their own thing with SELinux with the NSA and Google.

Do people see this as something though which should be or needs to be replaced or is this just a it's dead? If the first, are there any projects I could contribute to or is a new project needed?

What to use instead of SE Linux?

While I have nothing against SELinux, I feel that modern solutions using containers have made it almost irrelivant.

And now some SE supporters are fighting back in the dirtiest way possible. https://opensource.com/business/14/7/docker-security-selinux Mr. Walsh makes all sorts of claims about things, such as devices, which are never exposed to the container, not being namespaced. It is very bad when someone who is respected and has qualifications goes and spreads lies and insinuations, just because their product is no longer relevant.

I thought that Docker containers were primarily intended for environment isolation, not for security?

As I understood it, Docker relies on the kernel enforcing syscall namespaces for security... and that there's a (admittedly small) possibility of container breakouts via this route, because of the large surface area involved.

Even the Docker devs have a whitepaper that recommends running Docker inside a VM when multi-tenancy is involved, as part of a defence-in-depth strategy:

https://d3oypxn00j2a10.cloudfront.net/assets/img/Docker%20Se...

Docker uses SELinux, seccomp and apparmour as well as stock Linux capabilities (in conjunction with namespaces) to reduce the attack surface exposed to a container.

There's nothing inherently insecure about containers (look at FreeBSD and Solaris). The issues you see with Linux "containers" are usually caused by leaky abstractions (namespaces that don't namespace the entire kernel) and insufficiently clever setup scripts. But it is getting better.

One thing to watch out for when reading anything about container/Docker security is the date of the advice. that paper is from March last year, and there have been a lot of improvements to Docker security since then.

Docker makes use of namespaces/capabilities/cgroups and syscall filtering by default and you can add apparmour or SELinux policies on top if you like.

So the attack surface of a running docker container very much depends on how you've configured it.

It's likely still larger than a VM Hypervisor, but I know that there are large organisations using containers for multi-tenancy, so I guess some people are ok with that kind of configuration...

The surface area for both technologies is the same: The kernel. Howerver, SELinux uses a complex set of rules for determining access rights, while containers use basic bindmounts.

The issue is that local privilege escalation to root will always be a thing, even with seccomp limiting syscalls visible in a container. sVirt (SELinux as it relates to VMs and containers) can make sure that even with unintended root escalation you don't touch what you weren't intending to touch.

Full virtualization is preferred if this is a concern, trading off some overhead for increased isolation. But if the performance margins are thin for your containers (thus motivating the expected use of newer syscalls with increased likelihood of undiscovered/undisclosed escalations) and you're running something potentially adversarial, then take advantage of sVirt as a second layer.

And this kind of crap is why I use Windows.

Microsoft brings its own problems in. While it's true that NT's security model is much stronger than the Unix style (no ACLs on anything??), MS has really gone overboard recently. Install IIS and see the myriad things to check off. Dynamic compression is separate from static compression and you must install them apart from IIS alone.

The absolute best example: You cannot listen to HTTP without extra permissions. You can bind to port 80 and parse HTTP yourself. But if you ask Windows to do that for you, it's access denied. This is entirely illegitimate and many people just deal with it by running with admin permissions, rather than screw around trying to setup ACLs on HTTP prefixes. (The only context in which it makes sense is port sharing, so as to register certain prefixes. But in that case the port'll already be in use.)

Depending on OS flavor UNIX has had ACLs piled on top of the classic owner-group-other security model for decades.

SELinux in particular has decent ACL capabilities. By default Apache's policy allows listening on port 80 and a few other well known ports - you can configure the app to listen on $RANDOM_PORT but all that'll get you is an error in a logfile until you use semanage to fix it.

Or did I misunderstand your statement about no ACLs on anything?

I mean the default Unix model, like filesystem permissions, just have the very limited owner-group-everyone bits, nothing serious. SELinux does add a lot in but it's more of a bolt-on right? Compared to Windows where every thing, even an individual thread or other objects, come with ACLs.

Linux supports POSIX ACLs, which I would classify as "default Unix" since it's POSIX, and also supports some extended non-portable ACLs:

http://linux.die.net/man/5/acl

http://www.vanemery.com/Linux/ACL/POSIX_ACL_on_Linux.html

Ah yes, Windows, where you wake up one day and have a completely different OS.

Funny, I'm still on Windows 7 - I know which "updates" are related to the Win10 upgrade thanks to the well-written Microsoft kdb articles (not sarcasm), and I disable them. No problem here.

That whole one time leading into 95 (b.")b

I detest Windows and touch it as rarely as possible, but this week I had to make an emergency trip to a client's office to re-share a folder after a server had "upgraded" itself from 7 to 10.

Really hoping that was sarcasm.

Nope. I'm greatly discomfited that my computer browses a UNIX/Linux backed internet - I'd rather stick to those sites backed by Windows.

If he calls himself a security aware sysadmin but he is somehow breaking his own beliefs for productivity/delivery reasons, I don't see much of an interest.

Let him rant all he wants on his blog. Back to work.

Reply on news.ycombinator.com