LocalStack and AWS Parity Explained | Hacker News Reader

LocalStack and AWS Parity Explained

localstack.cloud

109 points_harshcasper36 comments

I used LocalStack quite a bit at my last gig. It sure is nice to be able to use Terraform to spin up the same infra on my laptop, and test against that. It reduced the number of developer sandboxes we needed to have in AWS. I sure do wish AWS would go the GCP route and offer an emulator. I prefer to use that in E2E automation that's run in CI/CD pipelines.

any chance you can point to a good overview on how to do this? im new to terraform and i'm trying to figure out the best way to co about local/sandbox testing

Really, all the skills of Terraform will apply. LocalStack's docs should be able to get you started: https://docs.localstack.cloud/integrations/terraform/

What's the difference between what localstack provides and an emulator as you describe? Just support, or do they do different things?

LocalStack is a third party that has a free and paid version of their product. I'm certainly NOT against anyone trying to make a living off of their own hard work but, Google provides their emulator free of charge. Supposedly this can drive greater adoption.

GCP emulators cover only a very small portion of their services though. AWS also has emulators for some services they provide free of charge. LocalStack adds more than just emulation though, it has a bunch of developer tools that enables local cloud dev much more broadly.

they provide a local dynamodb implementation...i dream of a day when they offer the rest.

Can you actually spin up local "ec2" vms with localstack? If so does it go as far as launching a KVM instance?

localstack pro will spin up a container as if it was an ec2 and it will act like an ec2 instance as far as localstack is concerned.

This is really impressive.

I've been running a scraper against the output of the "aws --help" CLI commands for a few months, to try and get a better feel for how often AWS changes - I call this "help scraping". The answer is it changes a LOT - there are updates to their APIs every single day.

Here's the commit log of changes I've tracked so far: https://github.com/simonw/help-scraper/commits/main/aws

https://awsapichanges.info/ will interest you, as might

* https://github.com/z0ph/MAMIP#readme

* https://github.com/SummitRoute/aws_breaking_changes#readme

I thought there was a backing github repo for AwsApiChanges.info but I was unable to readily locate it

The scope of this project is very impressive. I don't have a need for it myself at this point, but kudos for having even a semblance of parity with AWS for local development.

How does LocalStack compare to something like serverless[0], which lets you also have locally run abstractions over services?

While I realize the most obvious difference being that serverless is also a deploy framework that "abstracts" the cloud, I think one of its primary benefits of adoption is that it also does a good job (in my experience, but I have not yet impelemented things deeply with it) emulates services very well where needed, for instance, DynamoDB[1]

If it was without that, I think it would be a lot less useful and way less easier to adopt, so I think its just as important to the story.

Why on earth AWS doesn't have their own first party emulators for everything I still don't understand to this day. I credit that for why Firebase & GCP are easier to use, because they have a good local development story for alot of their services (Firebase in particular has an emulator suite for nearly all their services)

[0]: https://www.serverless.com/

[1]: https://www.serverless.com/guides/dynamodb

Great point. Fully agree that providing a first-class local development experience is critical to the overall story of application development frameworks.

I guess the main difference is that frameworks like Serverless provide a great experience if you fully buy into their way of doing things (i.e., implement your application assets in the Serverless YAML DSL, etc), whereas LocalStack is a generic platform that works on the API emulation level, hence easily integrates with most tooling out of the box.

Making the switch from Serverless to, say, AWS CDK, or AWS SAM, or Architect framwork may not be as seamless - however, for each of these frameworks you can always run the local emulation natively on LocalStack. This can help reduce the overall vendor lock-in effect that a lot of application development frameworks come with.

In fact, LocalStack also provides an integration with Serverless [0] - among many other tools [1].

[0] https://github.com/localstack/serverless-localstack

[1] https://docs.localstack.cloud/integrations

I'm biased but for local development, I prefer to emulate the least amount of AWS services. You don't have to deal with any gotchas while going to prod.

Some background, with SST we connect your local environment to the services deployed to AWS and just run the Lambda functions locally: https://docs.sst.dev/live-lambda-development

SST has been a breath of fresh air. Very excellent devex for companies using serverless.

Localstack is super useful for local (or CI build) development of tech that would be deployed into the cloud. I would definitely not recommend using Localstack for anything in production. I don't think serverless has anything quite as fully featured as what Localstack is able to provide, 1:1 parity across most of AWS's offerings.

I use it for stuff like SecretsManager and Cognito testing (as well as for S3). I don't use it for RDS emulation at all, and just stick a stock postgres container in that spot to achieve local development.

> I think one of its primary benefits of adoption is that it also does a good job [...] emulates services very well where needed, for instance, DynamoDB[1]

Under the hood, serverless uses DynamoDB Local which is the same AWS-provided DynamoDB emulator that LocalStack uses.

> Why on earth AWS doesn't have their own first party emulators for everything

Because they want you to pay for local development, not only production.

Google offers an emulator because otherwise no one would and they need to catch up with AWS, which has, what, 4x their market share?

AWS offers local emulators for services they need to catch up, like Lambda and DynamoDB.

I was excited when you said local Lambda emulator, but SAM just looks like a wrapper around https://github.com/lambci

LocalStack is cool, but don't forget that you always have an alternative to build mock infrastructure in front of your AWS deps. Don't call S3 deps directly - implement a higher abstraction using only the APIs you actually call and back it with your local filesystem (for example).

It's not a _lot_ of work, and making your dev/prod differences more explicit can make debugging much easier.

Or just throw minio on a box somewhere

It's work unrelated to your project, doesn't add any value.

Besides, it's reinventing the wheel.

And it's certainly cheaper to run local dev/tests against the real AWS S3 infra than paying the time to mock it with your local filesystem...

I disagree about the "doesn't add value" part when talking about introducing an abstraction over the AWS sdk: with mocks, it allows one to simulate failures, hung connections, and reproduce bugs without trying to contort an actual AWS account into the bad configuration that one might encounter in the real world

Your assertion about "cheaper" is also only true if one remembers to tear down all those resources when finished testing against them

Would be great to have a single table that shows supported services vs. tiers. At the moment I have to scroll through 8,000 services AND their features to try and decipher if I'll need a paid plan right from the start.

Does LocalStack allow me to actually mimic my entire infrastructure and services on AWS? Say I have a VPC with public/private subnets, a NATGW, ECS on EC2, ELBs, etc — can I mimic this infra in its entirety on LocalStack?

You can definitely emulate parts of this. For example LocalStack can spawn ECS tasks using Docker, or EC2 VMs using something like VirtualBox. Load balancers are also emulated, so you can test load balancing configurations with IP address or Lambda targets. LocalStack is currently not emulating all of the VPC network configuration, so although you can make the API calls to create VPCs subnet groups and so on, LocalStack currently wouldn't enforce these on a network level.

Thanks for the detailed reply. Would I be able to take most, if not all, of my existing Terraform code and use it to stand up my topology in LocalStack? If so, that would be quite useful.

yes, if you pay for localstack Pro, and "maybe" if you don't. they have a list of services they emulate on their site.

Here is the list: https://docs.localstack.cloud/aws/feature-coverage/

Thanks very much!

I admire the goal of localstack and have used it a few times in the past.

I always prefer tools that also work locally for development.

However, especially for software development there is more than just API parity. You'll end up with many unexpected behaviours when developing in a fully mocked environment. I doubt it will always behave like the real AWS

Reply on news.ycombinator.com