62 karma · joined November 11, 2022
If you want to reach out - i.a at the URL above.
There are mitigations you can put in place by using containers, virtual machines or even the execution environment e.g. Deno's ability to block/whitelist network calls[0], Bun's --ignore-scripts [1] and supply chain package managers have made some strides here like pnpm [2]. But it's knowing your threat surface and how to use your tooling which can be quite overbearing on cognitive load, especially in fast paced scenarios like "job of a lifetime offer!" from linked in.
Easiest way by default is to use ephemeral VMs / Sandbox Containers for such tasks which don't have mounted directories to your system etc. Or spin up a cheap EC2 / VPS to work on them in a short period of time.
[0] - https://deno.com/blog/deno-protects-npm-exploits and https://docs.deno.com/runtime/fundamentals/security/
[1] - https://bun.com/docs/pm/lifecycle
[2] - https://pnpm.io/supply-chain-security
[2] - https://
Dependencies and supply chain attacks are probably the greatest risk to a lot of software orgs, as they run them across all their environments: Development (with secrets and other valuable artefacts on developer VMs), CI/CD pipelines which may have access tokens to production (and other) environments, and production itself.
Notably even security companies are being impacted by this[0]. The scale of these attacks has amplified quite significantly the past three years, but are not solely exclusive to the javascript ecosystem [1] or even just namesquatting/typosquatting [2].
The resolution is broader security awareness, "onion layered" security controls and implementing simple non-burden inducing processes and policies. Sometimes not updating (what was wrong with the previous version of a dependency if there was no immediate vulnerability or production issue caused by it?) or having a two week cool down for updates (which some supply chain tooling natively supports) can appease some security functions through clear communication of the supply chain risk etc.
If anyone has interest in courses aligned to your org on improving developer and broader engineering management awareness on this, e-mails in my profile :).
[0] - https://socket.dev/blog/ongoing-supply-chain-attack-targets-...
[1] - https://orca.security/resources/blog/hades-pypi-supply-chain...
[2] - https://checkmarx.com/zero-post/python-pypi-supply-chain-att...
On a more serious note, the performance of hardware today is mind boggling from what we all encountered way back when. What I struggle to comprehend though is how some software (particularly Windows as an OS, instant messaging applications etc.) feel less performant now than they ever were.
Then you just query from event_receiver_svcX side, for events published > datetime and event_receiver_svcX = FALSE. Once read set to TRUE.
To mitigate too many active connections have a polling / backoff strategy and place a proxy infront of the actual database to proactively throttle where needed.
But event table:
| event_id | event_msg_src | event_msg | event_msg_published | event_receiver_svc1 | event_receiver_svc2 | event_receiver_svc3 |
|----------|---------------|---------------------|---------------------|---------------------|---------------------|---------------------|
| evt01 | svc1 | json_message_format | datetime | TRUE | TRUE | FALSE |
Do what you can to triage and see what's happened. But I would strongly recommend getting a professional outfit in ASAP to remediate (if you have insurance notify them of the incident as well - as often they'll be able to offer services to support in remediating), as well as, notify AWS that an incident has occurred.
[0] https://www.reddit.com/r/aws/comments/119admy/300k_bill_afte...
Based on docs and some of the concerns about this happening to someone else, I would probably start with the following:
1. Check who/what created those EC2s[0] using the console to query: eventSource:ec2.amazonaws.com eventName:RunInstances
2. Based on the userIdentity field, query the following actions.
3. Check if someone manually logged into Console (identity dependent) [1]: eventSource:signin.amazonaws.com userIdentity.type:[Root/IAMUser/AssumedRole/FederatedUser/AWSLambda] eventName:ConsoleLogin
4. Check if someone authenticated against Security Token Service (STS) [2]: eventSource:sts.amazonaws.com eventName:GetSessionToken
5. Check if someone used a valid STS Session to AssumeRole: eventSource:sts.amazonaws.com eventName:AssumeRole userIdentity.arn (or other identifier)
6. Check for any new IAM Roles/Accounts made for persistence: eventSource:iam.amazonaws.com (eventName:CreateUser OR eventName:DeleteUser)
7. Check if any already vulnerable IAM Roles/Accounts modified to be more permissive [3]: eventSource:iam.amazonaws.com (eventName:CreateRole OR eventName:DeleteRole OR eventName:AttachRolePolicy OR eventName:DetachRolePolicy)
8. Check for any access keys made [4][5]: eventSource:iam.amazonaws.com (eventName:CreateAccessKey OR eventName:DeleteAccessKey)
9. Check if any production / persistent EC2s have had their IAMInstanceProfile changed, to allow for a backdoor using EC2 permissions from a webshell/backdoor they could have placed on your public facing infra. [6]
etc. etc.
But if you have had a compromise based on initial investigations, probably worth while getting professional support to do a thorough audit of your environment.
[0] https://docs.aws.amazon.com/awscloudtrail/latest/userguide/c...
[1] https://docs.aws.amazon.com/awscloudtrail/latest/userguide/c...
[2] https://docs.aws.amazon.com/IAM/latest/UserGuide/cloudtrail-...
[3] https://docs.aws.amazon.com/awscloudtrail/latest/userguide/s...
[4] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credenti...
[5] https://research.splunk.com/sources/0460f7da-3254-4d90-b8c0-...
[6] https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_R...
Outside of laptops, Beelink and co. are making NUCs with them which are relatively affordable!
I do agree, the scarcity has limited their opportunity to assess the growth opportunity.
For data sustainment if they need to upgrade / replace VMs, have a secondary EBS (volume) mounted which solely stores persistent data for the account.
[1] https://learn.microsoft.com/en-us/windows-server/identity/ad... [2] https://learn.microsoft.com/en-us/entra/identity-platform/ms...
Provisioner scripts can effectively be broken down from the Dockerfile RUN layers as Bash or Powershell scripts. Then output as a vagrant box or hypervisor VM export VM of choice. For exposing ports, just run an iptables or powershell New-NetFirewallRule.
Whilst you can't get volume mounts can use the file provisioner to embed local files on build.
Then import it and create a default snapshot. Want it back to its original state? Revert snapshot and done!