Localstack – Local AWS Emulator
github.com
github.com
I know of a lot of big shops that are desperate to get out of the cloud due to 7-8 figure AWS bills but their software and engineering is too tightly wound into AWS tooling.
Yes there are open source public/private cloud alternatives like OpenStack but they won't solve the problem of lock in and you'd still have to change all your tooling.
That said before I found Kolla I did deploy openstack manually a few times before messing things up and having to restart.
Even "just" to emulate a single region with 3 AZs would be useful for testing and for prod for many people. I know of several large organizations that to this day haven't scaled their CI beyond one region.
Same deal with Azure and GCP, although note that Azure has Azure Stack, which is a self-hosted version of Azure (I guess it's still expensive though, and I don't know how many services it includes).
"Setting Up DynamoDB Local (Downloadable Version)" https://docs.aws.amazon.com/amazondynamodb/latest/developerg...
> The downloadable version of DynamoDB is intended for development and testing purposes only. By comparison, the DynamoDB web service is a managed service with scalability, availability, and durability features that make it ideal for production use.
"...Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow..."
Conflating lock in to an open source tool with lock in to a vendor who you literally have to keep paying money to survive is a shitty tactic used by companies like Amazon.
Using open stack absolutely solves the problem of lock in from a business perspective. You can’t be cancelled, the price can’t be changed, etc.
Just because something is open source does not mean your org will actually have the expertise to be successful at running it yourself. That's how most "open source" projects like Kubernetes and OpenStack become so successful: the supporting orgs behind the project make lots of money charging companies for support and maintenance and issues that pop up with the setup.
There are many forms of lock in. Buying a big upfront investment that has to pay itself off for the next X years is a form of lock in. Paying for expensive professional consulting support to build out the system and keep the system up and running is a form of lock in. Or even worse, hiring or training expensive dedicated employees to learn the open source system and become the internal professional consultants for the rest of the org, that is also lock in.
Here’s the part where it’s not lock-in. We stopped paying for support from Redhat when we got big enough that our own SRE expertise to run our own open stack cloud.
You know what the impact was on our applications running on it? Absolutely nothing.
How much do you pay Canonical to use grep every month? Inexperienced developers have just been conditioned by cloud providers to think that an IaaS platform must cost something in payments per month to some tech company. It does not.
This is no different than Windows vs Linux on servers. I can’t wait until 20 years from now when all of the proprietary shit looks ridiculous in hindsight.
I agree fully! As you explained, it can cost several full time employees and their payrolls and their HR and their management!
Edit: And before someone misses the point, I'm not saying the math never makes sense for in-house, but to me the inexperienced take is thinking either approach should obsolete the other...
> I can’t wait until 20 years from now when all of the proprietary shit looks ridiculous in hindsight.
I'd posit that in 20 years there will still be tech which is more efficiently managed by a central provider, rather than having each company hiring their own independent expertise.
Of course every solution costs something. Lock-in means that there is no other choice, or that the cost to switch is so steep that it becomes prohibitive to do so.
I share your disdain(?) for openstack, but the 12 node count there is minimum I think (to deploy the control plane). You can likely add a reasonable number of hypervisors at no cost.
(I think I'm correct based on the table in the last page of your doc, but happy to be corrected).
See how your comment ages in five years with kubernetes.
I've absolutely seen disasters with open source that bleed companies dry. Openstack is a great example.
Datacenters are fucking expensive to build and run, buying network and compute hardware is a pain in the ass, maintaining sufficient capacity to sustain growth while operating on a 3-6 month deployment lead time is atrociously expensive, and that's without all of the financial calculation around depreciation/taxes/etc.
The fact that cloud providers are able to ball all of that shit up into a flat opex rate for a server is unbelievably attractive.
[0]: https://docs.ansible.com/ansible/latest/user_guide/playbooks... Intro to playbooks - Ansible Documentation
CloudSeed has been a key part of several >$1bn contracts for KnightPoint/Perspecta/Peraton, most recently DHS DCCO ($2.7bn).
All that's old is new again.
I guess the (business) reason is that the centralized computing model enables enormous economies of scale. (Makes perfect sense from an operations point of view.)
What it seems to miss in the current incarnation/iteration, though, is the "power of distribution" - leveraging the fact that local compute capabilities (especially during development/integration) can help reduce the cognitive load, increase the efficiency, and thereby reduce overall costs.
There seems to be a tendency to focus mostly on the operational aspects, rather than the overall end-to-end developer journey. What remains to be seen is whether future iterations of "the cloud" will do a better job of embracing the power of distributed and/or hybrid.
Now it's exactly the same thing in AWS. My next guess is that you're going to be running production code on fleets of devs computers because you don't have to get extra budget for AWS next financial year to afford spinning up another instance.
The company works because every night there’s 1000s of engineers sleeping with a pager by their head.
we have often delayed calls because it's often hard to explain it to some "foreigner"
The way I made systems/applications before AWS existed was to create bootable images. I could already use a hypervisor, Xen, because the OS fully supported both guest and host, all virtualisation modes, before Linux did, and before AWS existed. But because I am not a hosting provider I saw little need for virtualisation. The OS I used also had "unikernel" capability, before Docker, etc. existed. As it happened, this inexpensive, self-determination was not to be the future, instead we got "the cloud". Sharing servers with other "tenants". Less expensive for the hosting provider, but more expensive and more limited for the subscriber. No argument, the "limited" part has improved since then, but it is still expensive (for continuous use that is, spot pricing was a neat benefit of "the cloud").
Anyway, I "deploy" images to "local bare metal" which is a laptop, RPi, or some other smaller form factor. I can use unikernel for some kernel drivers in userspace. Basic filesystem is embedded in the kernel, and larger filesystem is on the USB stick filesystem in compressed format. Updates are easy enough. I put two kernels on the USB stick, one is the running kernel and the other is the update kernel. Same for the larger filesystem which may contain servers and configuration files. Can update or go back to last working kernel/configuration by selecting one or the other in the boot menu/renaming the larger compressed filesystem.
Here is someone running a search engine out of his living room. AKAIK, his setup survived a sustained HN thundering herd, hug of death without a hiccup.
https://news.ycombinator.com/item?id=28552805
The expense of AWS is an obvious point of discussion but another one not mentioned here is control. When I create images for bare metal I do not need to jump through any hoops as I would in order to create an image that will run on AWS. Nor do I need to fiddle with all the AWS knobs. There are no silly marketing names for every program I run. I know the system I am creating as well as I know the OS and the software I choose. That is much better than how well I know every aspect of AWS which just gets more and more complex every year. AWS documentation is as cringeworthy as it it is voluminous. The ever-increasing complexity of AWS, including the "tooling", is, IMO, How To Create Lock-In 101
That is an interesting point. Complexity creates lock-in. Why? Because when you are interacting with a complex system you depend on it working in the complex ways it does. It is unlikely that anybody else could duplicate those features of the AWS your application is depending on.
This all runs counter to the idea of "encapsulation". You should be able to use a system via a well-defined interface. Once the interface is well-defined, other providers can provide their own implementation of the same interface.
So, AWS is basically bad software engineering, lacking encapsulation?
We were heavy EC2/RDS users, now we run their NX whitebox nodes with their KVM based hypervisor, and their database management platform "Era".
Everything is one click for deployment and upgrading of things, down to stuff like upgrading the SSD firmware, NIC firmware, etc. It auto migrates the VMs around hot, and does one node at a time. They also have a kubernetes platform called Karbon that seems to be pretty good.
I’m pretty sure it ended up where all software goes to die: HP.
- https://docs.eucalyptus.cloud/eucalyptus/5/install_guide/int...
- https://github.com/corymbia/eucalyptus/
There are quite a few moving parts. I think I got stuck around just comprehending the networking bits.
https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...
Happy for any additional suggestions and questions you may have. Looking forward to getting your feedback!
The main differentiator is that moto is a Python library that mocks out / implements the AWS APIs, whereas we see LocalStack as a platform that aims to support the end-to-end development lifecycle for your cloud apps.
A lot of the work we do in LocalStack is to create a seamless dev experience in your local environment - providing local DNS integrations, persistence features, Lambda code mounting, CI integrations, transparent injection of "localhost" endpoints into AWS SDKs (e.g., for your Lambda functions), and much more. Also, today we already provide some fairly sophisticated integrations - e.g., our Athena API which allows you to run your SQL-like big data queries natively over the local S3 filesystem. This is out of scope for moto, but a big focus area for us.
Our aspiration is that you can take any (AWS) cloud app and deploy it natively to LocalStack - which is already the case in many scenarios, and improving on a daily basis.
Btw, we're currently working on revamping/polishing our docs with lots of more content and details - we'll push out an update to https://localstack.cloud/docs in the next couple of days!
Why not have CI pricing per project/repo. Regardless of how often it runs.
For example, we're working on a new feature that will make it easy to snapshot the state of the LocalStack instance during and after each CI run, making the state "browsable" in the UI, and providing advanced insights and analytics into how the application stack and tests evolved over time.
We're also looking into alternative options for pricing - your point is well taken and will be considered for our roadmap. We definitely want to encourage testing frequently and on every commit - but I'd like to emphasize that our current CI offering is just a starting point, with lots more exciting features to come soon.. Thanks!
they have local dynamo, but i believe that's the only official appliance they provide.
I think it's especially necessary for the proprietary products -- dynamo being a great example.
I'll take a moment to call out snowflake for this as well -- it's really frustrating to not be able to quickly (within milliseconds) and easily (i shouldn't need to get credentials from elsewhere in my org) setup & teardown databases while under test.
i think they'd be better served by letting me develop more quickly, so I can help my company grow, and presumably need (and have the budget for!) more resources in our production environment.
if i'm mistaken, i think it's reasonable to have some sort of built in time limit to how long the appliance will run for, or some other way of preventing it being used for non-development purposes.
I found it useful when updating some existing old ruby scripts we had. (Of course they had little to no testing before that.)
You invoke it via the command line
`sam local invoke [OPTIONS] [FUNCTION_LOGICAL_ID]`
Behind the scenes it is running an attached docker container and feeding you the output. I found that it works actually very well and mimics the cloud invocation environment perfectly.
The problem is that it stubs out any other services you run. So if your Lambda adds something to SQS or SNS or SES or S3 or anything else, it simply stubs those out. So advanced functions do get difficult to test locally. But This Localstack emulator may solve those problems (but I haven't used it yet)
Details: https://docs.aws.amazon.com/serverless-application-model/lat...
PS: Localstack is awesome! Also in combination with Testcontainers.
Another example to add to your list - the new AWS CLI v2:
> don't force me to download a 30MB binary with embedded Python interpreter baked in, please just give us a pip package instead!
;)
Most of the code is just a thin veneer over that information...
I guess it's generally hard to find the right balance, if you're catering to customers of different level of technical expertise/maturity.
(Btw, the previous comment was in reference to: https://github.com/aws/aws-cli/issues/4947)
However for my own projects, we consider local development like this an anti-pattern. IaC tools like Terraform/CDK/SST make it easy to spin up environments for each developer in the cloud. This may sound crazy at first but would recommend giving it a shot.
If you're fully taking advantage of managed AWS services the best development environment is running everything in AWS. There's definitely some friction but you reduce the "was working on my machine but not in production" situations.
But what do you test your IAC against? If you consider using something like Terratest, then combining it with LocalStack means you can run Terratest nearly instantly and test your IAC.
Plus developers being able to run "AWS" locally menas they, too, benefit from the instant feedback loop. That alone is a good enough reason to use LocalStack, but there are other massive positives: no credentials to a real AWS account to leak; no developers spinning up some new AI tool 'cos it looks cool; and more.
The feedback loop is definitely a problem but projects like SST return the instant feedback loop to you.
One improper setting in IAM and an accidental pastebin or GitHub means half the automated hacks have access to the same thing the IAM user does.
I did this once on accident by hitting control-V instead of control-c and overwriting my censored version before pasting into GitHub so other people could save the 4 hours of tedious scripting I had to do. GitHub sends an email within minutes, but I was already miles away from a computer when I got the email...
Paul Swail has spoken a lot about this and has an upcoming book around the concept: https://serverlesstestinghandbook.com/
For one, the article assumes the tools you use to replicate your production environment locally aren't accurate; with localstack, they are accurate. It's seamless, truly. I recommend you give it a try instead of spending the extra cash to have your developers deploy into the cloud constantly.
Another thing to note is that this seems only true for a specific architecture, if you're writing serverless functions. Not everyone is or should be doing that, so I'm not sure "you should always test in the cloud" applies as a "context free" rule.
Finally, it sounds like the author had a lot of trouble getting his local environment to work. Maybe folks who have a bit more experience working with the technologies involved in their services don't have that problem, and it's a little arrogant to assume that, just because you had a problem, that it's a problem for everyone else, too.
Overall, I just don't think commenting, "This service nice, but using it is an anti-pattern" was quite as productive of a comment, in general, as you meant it to be.
In turn, all of the AWS APIs are defined by a kind of JSON schema document, so there’s not really any reverse engineering going on.
I consider Localstack somewhat toxic in most projects because it shows the the developer didn’t understand how to contract test.
Furthermore, doing anything more than contract testing with AWS is a fool’s errand. The problem you face with Localstack is that it gives you an inaccurate model of how AWS actually works. As a general rule, AWS APIs are asynchronous, as in their side effects are not atomically bound to the request you’re making (sans some certain S3 actions); while Localstack is invariably synchronous.
I can’t really say if it pays off but it has caught some issues, would like to expand them more and implement more logic in then to check arguments etc but haven’t been able to yet.
This article is similar to what we’ve done
https://dev.to/elthrasher/mocking-aws-with-jest-and-typescri...
We also have a pipeline that does e2 work tests and canary releases to hopefully limit the blast radius. If we encounter >10 errors in a 1 minute period we roll back and the team are alerted
We're currently investing some time and resources on refactoring the code base to improve the performance. We have some improvements in the pipeline which we should be able to roll out within the next 1-2 weeks.
Performance (as well as general UX) is definitely high up on our priority list, and we'll continue to invest heavily in this area. Stay tuned!
Happy for any additional suggestions and questions you may have.
Have a couple lambda services backed by API GW that I’d like to have a better dev env for.
LocalStack takes a different approach by working on the API emulation layer - hence making it easier to switch and integrate with various different application development frameworks.
Btw - there's also a Serverless plugin, as well as a "samlocal" command line - in case you'd like to give it a try:
…And just trust that I destroy them before they incur cost.
There's also a "cdklocal" command line which should make it fairly easy to get started with local CDK development: https://github.com/localstack/aws-cdk-local
Btw - we're also offering integrations for Terraform (https://registry.terraform.io/providers/hashicorp/aws/latest...) and Pulumi (https://github.com/localstack/pulumi-local), among others.
We'll soon publish some more details on the TF integration in the LocalStack docs...
Side note - similar to the *.tfstate file maintained by Terraform (storing metadata about resource deployment states), we're increasingly making use of persistence files (called "local cloud pods") that allow you to easily store the state of your LocalStack instance, share it with team members, keep track of changes over time, etc. Really nifty feature...
https://cloud.google.com/sdk/gcloud/reference/beta/emulators
The firebase stack can run 100% local as well.