Comment by ptcrash | Hacker News Reader

Comment by ptcrash | Hacker News Reader

Parent Full thread

Okay, opinions up front: I don't think this is worthy of "declaring a security incident. Having some experience working behind the scenes, just because this policy was changes this way doesn't mean "All AWS Support personnel had unrequited access to your S3 objects." To me, this reads as Twitter inflammatory nonsense. Here's why:

* KMS Encrypted objects would not be accessible because the support personnel would need permission policies that grant `kms:decrypot` permissions to encrypted objects. The only way this could wind up happening is if you are granting the AWS Support principal access in the KMS Key Policy.

* Objects with a default-deny bucket policy could not have been circumvented with the support team's escalated privilege. So if you have a policy that looks something like this, that data was not exposed:

{

  "Action": "Deny",

  "NotPrincipal": [...]

}

* Internal Checks. AWS has a lot of protections and checks in place to prevent their support personnel from accessing metadata about S3 objects. They don't have tools to fetch the actual objects unless your really high up the food-chain. Think like, people with a legal or security related reason to need to review data.

Nevertheless, I'll share some nuggets of wisdom I've accrued over the years, in a hopes to save y'all some time:

If you have an NDA with AWS, I'd recommend reaching out to your TAM and asking them about what the potential exposure was; and make sure to ask about the internal access control mechanisms.

But everyone who's concerned and DIDN'T set up data access logging already: 1) Consider turning that on to trace potential disclosures in the future. 2) Open up a case with Premium Support under the CloudTrail, state that you have a security incident and you need to retrieve data events for the time of 2021-12-23 to 2021-12-22. If granted to you, save that sucker in S3 and query it for requests coming from `support.amazonaws.com` in Athena. [0]

Hopefully this helps some of y'all.

[0] https://aws.amazon.com/premiumsupport/knowledge-center/analy...

"Declaring an incident" means there's something to investigate, it didn't mean anything bad has happened: it's detection of a non-conformity. The output of the incident would look similar to what you wrote.

Any time the wrong permissions are assigned and confidentiality is potentially breached, I think you have to have an incident. Arguably in some jurisdictions, it's a legal requirement to ensure you have a near miss not an actual breach.

Exactly. This is something we'll be reviewing in our monthly security review at work, discussing what the impact was, why we were not impacted, and any action items we want to take.

Declaring an incident doesn't mean sending out a breach report or anything particularly dramatic, though I can see how as an outsider it may sound that way.

This is partially right. The best way to think of an incident is that it has negatively impacted Confidentiality, Integrity, or Availability (the CIA triad).

An incident does mean something bad has happened, and requires action. An example of an action can be to investigate the impact, or to shut something down, or to patch something.

That's enlightening; thanks. Hopefully the steps forward for determining scope and affected objects is easy

> * KMS Encrypted objects would not be accessible because the support personnel would need permission policies that grant `kms:decrypot` permissions to encrypted objects.

This is only true if you are using SSE-KMS *and* are creating/managing the CMKs used to encrypt objects. If you’re using SSE-S3 or SSE-KMS with the default AWS S3 key (aws/s3) there is no key policy to manage.

Of course SSE-C or 100% customer-managed crypto would be immune as well, but under different mechanics.

> Objects with a default-deny bucket policy could not have been circumvented with the support team's escalated privilege.

I would wager this is done for a vanishingly small percentage of buckets used in production. Less than one percent for sure.

The general point you’re making seems to be that if you had a comprehensive, defense-in-depth security strategy for your cloud computing environment then this would have had no effect, and i do agree with that. I just think that in reality this would have provided access to wide swaths of customer data.

AWS uses envelope encryption. Data encryption (for example bucket-level) keys are generated and managed outside KMS for some time/services to minimize calls to KMS that can be expensive.

Thus, I imagine with AWS-managed KMS keys, in some cases permission to KMS service is not needed to access data.

KMS-CMKs are better in this respect, can be controlled and audited.

> * Objects with a default-deny bucket policy could not have been circumvented with the support team's escalated privilege. So if you have a policy that looks something like this, that data was not exposed

Service accounts are not constrained by customer bucket policies. In fact, not even SCP's are restricted by service-linked roles:

"SCPs do not affect any service-linked role. Service-linked roles enable other AWS services to integrate with AWS Organizations and can't be restricted by SCPs."

https://docs.aws.amazon.com/organizations/latest/userguide/o...

Sure it is. AWS now needs to go on record as to 1) why they did this, 2) how much of this access was used, 3) who is able to use it, and 4) whether this was done at the behest of some government.

Reply on news.ycombinator.com