Hacker News Reader

Comment by mtam | Hacker News Reader

+1, however, from what I read, the vulnerability can only be exploited if the attacker has network access to the salt masters port, which should never occur. The people that got compromised had Salt exposed to the Internet, which is obviously ridiculous.

Not trying to downplay the critical nature of the vulnerability but the ones that were compromised by this issue have deeper security issues to deal with.

View on HN

> has network access to the salt masters port, which should never occur

You seem to prescribe to the "hard shell soft gooey center" network security philosophy. Should people expose an Oracle server to the internet? Absolutely not. Does moving it behind a firewall change the fact that every mildly skilled exploit developer is sitting on an Oracle 0day? Absolutely not.

People have legitimate reasons for exposing Salt to the internet. I do. It's how I bootstrap random VMs and bare metal from the internet. But in my case the attack was mitigated by the fact that Salt cascades changes in a bunch of other systems and re-masters minions to a host only reachable over a tunnel. I blew away the internet master, restored from a backup, and patched.

> the ones that were compromised by this issue have deeper security issues to deal with

Or it was just another Monday. When you become sufficiently large you deal with incidents on a daily basis. Kudos to the people who publicly postmortem and talk about what went well and what didn't.

(For the record, I've already been working for a few months on a move to Ansible for non-security reasons)

> People have legitimate reasons for exposing Salt to the internet. I do. It's how I bootstrap random VMs and bare metal from the internet.

I question that that is a legitimate reason to expose it to the internet.

Defense in depth is a thing and putting the keys to the kingdom at layer 0 doesn’t seem wise even if a vpn or bastion doesn’t offer perfect protection.

Read the sentence after the ones you quoted. The internet connected salt master is used to provision accepted hosts in to the tunneled (VPN) network where the real master lives.

You piqued my interest, what made you move? I personally haven't touched Saltstack in years, but was super happy with it when I did.

Twice I encountered breaking changes between versions that required manually upgrading minions. I also got the overall feeling Salt was built by developers, Ansible by sysadmins - and I fit into the latter bucket.

Ansible (originally known as "Fedora Unified Network Controller" or "func") was made for solving the problems automating Fedora Infrastructure.

Puppet did not make Fedora Infrastructure administrators happy. So func was designed around solving their problems, and expanded its scope as people found it useful. Then it was renamed to Ansible, the developers left Red Hat to create AnsibleWorks, and the rest is history!

So yes, it was made for sysadmins. :)

> It's how I bootstrap random VMs and bare metal from the internet.

What's stopping you from bootstrapping a bastion server through a simpler script and then bootstrapping with Salt from there?

That is exactly what the internet connected Salt master does. It bootstraps enough control that I can get the tunnels and keys properly configured, and the other 95% takes place once it is switched to a protected Salt master.

Coconut security is not great either. Hard shell, soft internals. Not exposing the ports to the internet is just one layer.

+1 agree but exposing salt to the internet is not the problem. A simple ip whitelist ingress firewall rule on the salt master port would have helped, blocking access is also possible on this port. With cloud services it has become trivial to group server resources so that when they belong to the same group they can communicate with each other. I don’t use salt however i am not a proponent of network isolation as a form of security.

that’s just plain negligent. of course you use network isolation as one of your defenses in depth.

I was in this situation; I went with “salt master exposed to the internet” because it’s the only service on that box - if I’d wrapped it in a VPN, then I’m replacing one exposed service with a different exposed service, and VPNs aren’t immune to exploits either (plus an extra layer of configuration means an extra layer of things that can go wrong)

If they wrote software which should never be visible to the internet, they should have made that clearer.

It's far too easy to make something internet-visible. They could have set up a simple check to see if the service is internet, and refused to work if it was.

If you look at their current `hardening` document it still has pretty unclear language about what is acceptable and what isn't.

> Use a hardened bastion server or a VPN to restrict direct access to the Salt master from the internet

Is this SSH access or is this access to the salt master from minions? Or just access in general?

Apparently it includes minion-master interaction. If that’s to be “hardened” over SSH, what’s the point of all the salt keys?

SSH bastions and VPN are two standard ways to allow external clients access into an internal network, meaning salt is never exposed publicly.

I read this as a guideline that the salt master must not be exposed to the internet. Albeit could be better worded for a developer audience who doesn't understand bastions or VPN well.

Reply on news.ycombinator.com