Switch to VPC Endpoints from Nat Gateways to Reduce Bandwidth Charges
vantage.sh
vantage.sh
I built my own NAT AMI[1] that costs <$4/month and has zero bandwidth charges and is good enough for most people.
"But what about AWS' NAT Instance AMI?"
The official AWS supported NAT Instance AMI hasn't been updates since 2018, is still running Amazon Linux 1 which is now EOL, and has no ARM support, meaning it can't be deployed on EC2's most cost effective instance types. fck-nat.
To be fair, big tech despite their massive volume pay much higher rates than small networks because the carriers charge them enough to fully cover their costs to build out their networKs, while they make all their profits from selling their excess capacity to the little guys for pennies on the dollar.
*A long, long time ago, I looked at about 1000 co-location customers' MRTG stats and compared their monthly 95th percentile Mbps to their average sustained data transfer in GB, and something like 90% of them were between 150GB-250GB per Mb and 98% of them were between 180GB-220GB. Many people assume 324GB which would require their traffic to be perfectly flatlined throughout the month, which obviously rarely ever happens.
You're one misconfigured security group away from your shit being owned.
Also even if there's no firewall at all, how does that mean your machines are getting owned? My boxes listen on precisely one port: a heavily locked down sshd (which isn't listening on that interface anyway)
> listen on one port ... which isn't listening on that interface anyway
Would you mind please elaborating, here? And which interface does sshd not listen on?
This was pretty confusing to learn at first.
Amazon has most of an example here using quagga (but CloudFormation, ick): https://github.com/aws-samples/aws-transit-gateway-connect-s...
is there any practical mechanism by which not using the VPC endpoints can be insecure, which does not also affect VPC endpoints?
i usually get some hand waving about man in the middle attacks, but i figure if you can insert that into an AWS data center you can probably just see all the VPC internal traffic anyways. and one should be using TLS to talk to the services anyways.
Hum, yes. If you're not using VPC endpoints, basically you're routing all your AWS traffic to the open internet.
Not only is it wasteful and slow, but that also means you have to open free lunch internet egress on your instances, or implement some convoluted DNS based firewalling.
Using VPC endpoints also allows you to capture AWS service traffic and apply policy on it, such as whitelisting which buckets are accessible, thus avoiding internal data leaks.
Honestly there is just no argument for _not_ using VPC endpoints. Even on a purely architectural point of view, having you AWS traffic being routed out to internet just to get back in makes no sense.
If you are still unconvinced, the pricing argument of course still stands.
yes this is what i normally hear. but by “the open internet” you mean “some other spot in the AWS data center”. what’s the risk here? somebody is going to slip some bad routes into AWS over BGP? and then also fake the SSL certs? this seems like some mission impossible stuff.
> Honestly there is just no argument for _not_ using VPC endpoints.
unless something has changed in the last three months they’re not available for all services. would you advocate against using those services?
> If you are still unconvinced, the pricing argument of course still stands.
sure that and the exfiltration argument are reasonable enough.
thanks for response
Well I can only guess so much of the underlying egress internet routing of AWS. At worst, if no explicit region is specified, it will reach the global aws endpoint through internet which is likely in a complete different part of the world than where you are, redirect to the local endpoint, and back.
> what’s the risk here?
Minimal, though I'm not sure the question is really relevant.
It's a bit as if you design a house with no internal doors, and you have to get out the window and back through the front door whenever you want to change room. I guess that wouldn't make you house less safe, though it's definitely a design that smells weird.
There is a real security point to make on the fact that it forces you to open access to egress internet though, and that is not to be taken lightly. There is no reason to allow a server full egress internet, and accessing AWS through internet basically forces you to do so, or leaves you implementing DNS based firewalling which is error prone, less secure, and overall a pain to setup.
> unless something has changed in the last three months they’re not available for all services. would you advocate against using those services?
No. Though I would (and do) strongly recommend implementing either DNS based firewalling, or a dynamic ruleset based on AWS ip ranges (they publish it as JSON).
> At worst, if no explicit region is specified, it will reach the global aws endpoint through internet which is likely in a complete different part of the world than where you are, redirect to the local endpoint, and back.
There's no need to guess:
From https://aws.amazon.com/vpc/faqs/#Peering_Connections
"When using public IP addresses, all communication between instances and services hosted in AWS use AWS's private network. Packets that originate from the AWS network with a destination on the AWS network stay on the AWS global network, except traffic to or from AWS China Regions."
In practice there is not much risk from accessing AWS services using public endpoints, you just need to take AWS at their word.
This means as an attacker, I can also go spin up (s3/lambda/etc) and if your app was vulnerable in some way, I could exfil data to those aws services (or leverage them in RFI/CSRF/other attacks) in a way that shouldn't be possible.
> I could exfil data to those aws services
I don't think any of the VPC endpoints have any restrictions on them; that is, once I've made an S3 endpoint in my VPC, it can be used to access any bucket. So... it seems like it could be used just as easily to exfiltrate data?
On the contrary, and that is one of the strong points of using VPC endpoints. You can whitelister the buckets that are accessible through the endpoints, which means it is in fact the only way to prevent data exfiltration.
Edit, link: https://docs.aws.amazon.com/eks/latest/userguide/private-clu...
But EKS by itself, you can absolutely do with private subnets and private link API's.
However you still need something to do NAT if you are running an ipv4 private network and need to access the internet
Run a proxy / LB on an instance that has a public IPv4 and a VPC network interface. Don't NAT or route between these networks, just let the proxy listen on the public interface, and contact the backend servers on the private interface.
check the docs very carefully before getting your hopes up about dropping the bar gateways.