Ask HN: No clue how AWS account was compromised – how to move forward? | Hacker News Reader

Ask HN: No clue how AWS account was compromised – how to move forward?

16 points71a54xd5 comments

I recently got an email from AWS about an account I created for consulting work nearly six months ago. Someone managed to get in and stand up a bunch of EBS clusters. Fortunately, this was caught by logs at the startup where I was consulting via their analytics. This was an account I'd only logged into a few times with it's own separate email.

I'm sort of afraid to keep doing anything important or sensitive on the macbook where I logged into this account. Fortunately amazon has been great with support and it looks like I'm off the hook for the fees. But I'm in desperate need of advice for how to prevent this (even with 2fa) in the future and how to safely move all of my bitwarden credentials to a new completely sterile machine?

I guess I'm just a bit flustered since I've never had this happen before and I consider myself a solid developer with good security practices.

View on HN

The same thing happened to me a year ago. I had an AWS account I barely used except for an SQS queue. Somehow, someone got in, changed the password, and set up a machine learning pipeline. I couldn't turn it off since I was locked out of the account and I racked up a $20k bill.

I also have no idea how they got it. I had 2FA set up and only had one service key created that I used in a Heroku environment variable.

Here's what happened:

- I contacted customer service. It took them several days to get back to me. Initially they told me they couldn't help and I would be responsible for any charges per their ToS as it's my responsibility to secure the account.

- After some back and forth, they reset my account credentials (the email was changed from me@mycompany.com to uuid@random.ru so it was obviously an account takeover).

- They listed out a list of services that had been started after the compromise and told me it was my responsibility to disable them and then tell them I did so.

- I cleaned things up the best I could and then told the service agent. They said I missed a few things and gave me more clear directions.

- By this point I had a $70k bill. Things had been running for about a week.

- I asked about getting a refund and they said they could do that but only after I set my account up with a proper security setup, which involved creating a bunch of separate small user accounts with minimal permissions.

- I did that, they refunded the charges, and then I deleted my account.

Long story short, it took a while and they weren't initially too helpful but ended up being very nice and helpful in the end.

Imagine if they had not reimbursed it!

No kidding. They told me they wouldn't reimburse me if it happened again which is why I deleted my AWS account and moved to something else.

> By this point I had a $70k bill. Things had been running for about a week.

Jesus. This is terrifying.

> how to safely move all of my bitwarden credentials to a new completely sterile machine?

Bitwarden is synced so just wipe the machine and log in again?

Reply on news.ycombinator.com