Hacker News Reader

Comment by mcbain | Hacker News Reader

It is a such tiny thing, but I really like that the metadata service is at fd00:ec2::254

On v4 it is 169.254.169.254 hence the final 254 (and other services like DNS at 253), but using the ULA range of ec2 is a simple, effective touch.

View on HN

I really hate that the metadata service is reachable via IP at all, and not just via a file based unix socket or a similar endpoint. Or even similar to procfs, instead of http based at all.

You'd rather mount a socket or file to a container that hit the endpoint directly over HTTP?

I prefer the latter as it's far simpler.

The problem is the metadata endpoint gives any process on your VM that can make API calls to that address full AWS credentials for that instance’s role. That can turn out to be too simple to use.

...unless you use identity-aware firewall, such as iptables:

  iptables -A OUTPUT \
     -d 169.254.0.0/16 \
     -m owner -j REJECT \
     '!' --uid-owner 0

Sure, but what percentage of EC2 instances do you think have a firewall rule like that? Defaults are important. Not to mention the fact that the rule you listed breaks down if you’re running docker containers.

ECS and EKS have configurable functionality to blackhole IMDS. You can also configure IMDSv2 to have a hop limit of 1, preventing any bridged networks (such as containers) from accessing IMDS.

There’s no good defaults here. From AWS’s perspective the whole VM is in the same security context. It’s all untrusted customer code from the kernel on up. That’s the granularity on which policy can be applied. Either the whole VM can do something or it can’t. How you define your own security boundaries inside is your business. You can help AWS by doing your own access keys but most people would rather just also make their security boundary the VM and not have to think about it. Treat a compromised processes and a compromised host the same.

That's the "give up on container and process security" approach. It's a fine approach from any given user's standpoint. From an infra/tool provider standpoint, it ignores the intense demand that exists for establishing least privilege and defense-in-depth for containerized workloads. Fargate might be a good solution too.

This is backwards, you don't have to give up on container/process security just because AWS's own trust boundary is the VM. But to do that you can't assign VM-level privs. Something something eating cake.

Put yourself in AWS's position. Your customer is running a VM which you have only hypervisor level control over. Could be Linux, could be AIX, could be an appliance. How could you possibly implement user/process level security from the outside? How could you know what processes inside the opaque black box are the privileged ones?

Sorry, I don't think what you're saying makes sense. A key strength of AWS is that they don't use a "black box" approach when thinking about customer needs, and provide documentation and good defaults to help customers achieve their goals.

  > From AWS’s perspective the whole VM is in the same security context.

That's AWS's perspective. But my perspective, as an AWS customer, is important as well. And I might want to run something on a VM that I myself don't necessarily trust - that is the ultimate benefit of an ephemeral, isolated VM.

Right, then you run that code in a VM that has no ambient privs. If you still need to do AWS access stuff then you just stick your access keys in files somewhere your untrusted process can't touch like if you weren't on AWS.

Hmm, so root processes within the container can still access it?

Yeah, if you're running containers with root access or processes with root running inside of them, there are a number of other things that will need to change about your security posture, least of which is probably how you provide or restrict access to IAM roles within the tenant boundary.

No, recent docker uses UID namespaces, where "root" inside a container translates to a high real UID as seen from the host OS (100000 or some such). iptables' "owner" extension would use that real UID to match.

Which could be a problem if intention was to allow it, as your sibling comment points out.

Yes - this is not really a Docker thing but a Linux kernel thing, although client-side support is of course needed from Docker and any other system that uses cgroups/namespaces. Also, one other thing to know is from the kernel's point of view "root" is not a thing. It has been unbundled into a set of capabilities (https://man7.org/linux/man-pages/man7/capabilities.7.html). When you launch a container, you specify which capabilities you want to drop (so even root in the container can't have them), and which UID mapping you want to use.

That's the point of the endpoint, IMHO. It's a well known location where anything in the VM can locate the credentials.

IMDSv2 is supposed to solve that

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configur...

There are solutions for this: http://www.daemonology.net/blog/2020-01-27-Announcing-imds-f...

Unauthenticated HTTP is harder to secure against SSRF

When you use the AWS SDK, it abstracts it away anyway

A file based unix socket or procfs node is an operating system implementation-specific concept. It requires a substantial amount of logic to be bundled with the guest OS.

Maybe a serial device or something like that would be more appropriate.

These addresses technically violate https://datatracker.ietf.org/doc/html/rfc4193#section-3.2, unless "0x000ec20000" was an incredibly lucky RNG output.

But 254 in hexadecimal is 0xfe :/ 0x0254 is 596, or .2.84 in IPv4 notation.

You could write it as fd00:ec2::0.0.0.254...

The point is that the hex looks the same as the decimal. The actual address doesn't matter. They’re not doing for decimal 254. They’re going for “the last part of the address is familiar and typed the same on a keyboard”.

Sure. I was just pointing out that if you really wanted to use 254, you could use fd00:ec2::fe and write it as fd00:ec2::0.0.0.254 -- though mainly because I wanted people to know that doing so was possible, not because it was in any way a good idea in this case.

(At least, I was trying to point that out, but re-reading my post it's obvious that I didn't do a very good job.)

No I get it. It’s a fun fact.

Reply on news.ycombinator.com