Ansible, Chef are for building configuration, you know menus, staff schedules, grocery lists. Ensure a restaurant is configured correctly to serve customers with its wait staff (Ansible)
You can use an excavator to configure the stuff inside the restaurant. People do it. It's just not generally the most efficient way to do it. And you could have the wait staff at a restaurant pouring concrete for their second place a town over. You could do that too, but a lot of people would use the excavator.
So what really makes these tools effective is when you start using them at scale. They start to become helpful once you realize how much you can do with how little, and they each have this same strength solving different levels, and their strengths become weaknesses at the other end.
a)Ad hoc scripts
The most straightforward approach to automating anything is to write an ad hoc script. You take whatever task you were doing manually, break it down into discrete steps, use your favorite scripting language (e.g., Bash, Ruby, Python) to define each of those steps in code, and execute that script on your server
b) Configuration management tools Chef, Puppet, Ansible, and SaltStack are all configuration management tools, which means that they are designed to install and manage software on existing servers.
c)Server templating tools An alternative to configuration management that has been growing in popularity recently are server templating tools such as Docker, Packer, and Vagrant. Instead of launching a bunch of servers and configuring them by running the same code on each one, the idea behind server templating tools is to create an image of a server that captures a fully self-contained “snapshot” of the operating system (OS), the software, the files, and all other relevant details.
d)Orchestration tools Server templating tools are great for creating VMs and containers, but how do you actually manage them? Handling these tasks is the realm of orchestration tools such as Kubernetes, Marathon/Mesos, Amazon Elastic Container Service (Amazon ECS), Docker Swarm, and Nomad
e)Provisioning tools Whereas configuration management, server templating, and orchestration tools define the code that runs on each server, provisioning tools such as Terraform, CloudFormation, and OpenStack Heat are responsible for creating the servers themselves. In fact, you can use provisioning tools to not only create servers, but also databases, caches, load balancers, queues, monitoring, subnet configurations, firewall settings, routing rules, Secure Sockets Layer (SSL) certificates, and almost every other aspect of your infrastructure
> not only create servers, but also databases, caches, load balancers, queues
These things are, I would say unquestionably "infrastructure".
> firewall settings, routing rules, Secure Sockets Layer (SSL) certificates
These things are more or less configuration, basically files that exist on the above.
And yet, as you say, it's common to manage them using provisioning tools like terraform.
> These things are more or less configuration, basically files that exist on the above.
Part of the different perspective feels like managed cloud vs on prem.
In the former, these are all actually things to be created, albeit as an abstraction on the underlying implementation. Which you don't have access to.
In the latter, they're configurations on things you have access to.
Ansible is for 'inside' virtual machines or computers, Terraform is for 'outside' virtual machines or computers.
Inside a machine you might have software, configuration, assets. Outside a machine you might network connections, firewalls, disks, dns etc.
This isn't a comprehensive comparison, but when you start from nothing, it doesn't really help to do a syntax, provider or imperative vs. declarative.
See https://docs.ansible.com/ansible/2.3/list_of_cloud_modules.h...
Essentially you need to 1. do the checks to ensure your playbook won't just create a new set of VMs every time it's run if they already exist 2. maintain a teardown playbook alongside your setup one because Ansible is entirely procedural and the steps would be reversed in that case and 3. do queries first to determine what actually exists in the cloud and do lots of jinja manipulations to work on the right things. Did you know EC2 has default subnets and routing tables? Did you know that the Ansible module will error out if you try to delete those objects?
If only there was a thing like Terraform that could just rely on a single description of the setup you'd like. Seriously. There's an Ansible module that will run a Terraform .tf file, and there's a provider for terraform that will run Ansible on the servers it provisions.
I use Ansible to provision dev environments on each PR, it works fine, every time a deployment occurs Ansible will deploy if required, otherwise proceeded with the deployment.
Destroying the environment is done separate, triggered by a webhook once the PR is closed.
What I'm talking about is using the cloud modules to spin up servers, and god help you, entire VPC set ups.
If someone is new to this, having a strong boundary makes it much easier to process how those systems are intended to be used and how they are built. There are certain assumptions made in both that make them work differently, i.e. being command centric vs. resource centric. That's interesting when someone is up to the point where you can apply what you know in a more general or generic sense, because then he systems themselves matter less.
https://www.hashicorp.com/resources/ansible-terraform-better...
TLDW; You can do resource management (e.g. creating EC2 instances in AWS) and deployment (e.g. installing packages on an instance) through both Terraform and Ansible. Terraform is best used for resource management - the documentation states using the "provisioning"/deployment function is a last resort. Ansible is great at deploying packages but less so at resource management for the reasons you'll see in the other comments. Either use them together for what they're good at, or use Terraform to do resource management and other techniques (such as prebuilt images) for deployment:
https://www.terraform.io/docs/provisioners/index.html
Also useful:
https://blog.gruntwork.io/why-we-use-terraform-and-not-chef-...
For example, you can use Terraform to provision virtual machines, database instances, or Kubernetes clusters on AWS. Terraform does this via the AWS API.
In my opinion, Terraform is better for provisioning because of the way it manages its own state. Terraform remembers what resources it created the last time it ran, and can edit or delete them according to any change in your Terraform code.
I like Ansible, but not for managing cloud resources. Ansible has no memory. For example, if I ran a playbook that installs MySQL, Ansible has no built-in way to undo this change and bring me back to my previous state.
If you (re)create some EC2 instances with Terraform. Terraform save the ID the first time they are created (in a state file that needs to be shared and keep in sync). It goes mental the next time it runs if any of the instances are not found, or the state file is missing, or some of the instances were modified or died.
Ansible always lookup what's actually running, instances with the intended name/tags and match versus what's expected. It skips when it's already there, it's much less accidentally destructive and never run out of sync.
Tracking AWS apis is a fulltime job and ansible for clouds just isn’t popular enough.
Terraform has better support for some static things, mostly VPC, routing tables, gateways. I've had infra retrofitted in terraform but honestly it's more for the show and as a documentation. Low level needs only be setup once and it's always been done manually forever ago.
If you were working around 2014-2017, both tools and many AWS services were new. There were significant gaps in support as well as a few bugs. Had to run from the beta build regularly. It is much better nowadays.
For example on AWS we use the s3 backend and a dynamodb table for locking. This way when terraform runs it will first acquire the lock, and then access the state on s3. And everyone is working on the same state.
IMO The way terraform automatically/accidentally delete stuff is a major design flaw, not a feature to emulate. It's madness that it tries to auto nuke potentially a whole company just because it lost track of one resource identifier.
That's putting it backward to say the least. One never tells terraform that something is not needed anymore. One declares what is needed and terraform will find a way to get there by altering/creating/deleting stuff.
There is a review phase of course and it's very important because it might do anything. Anybody who's had to use terraform can attest that it is scary to run. Any slight error in configuration or state can be tremendously destructive.
This is no worse than Ansible - if for a set of EC2 instances the user "set to 0 to delete them" then Ansible will blindly do as requested and be just as destructive. On the other hand:
* Terraform does its best to enforce the recommended plan/apply workflow - the plan is always presented before any changes are made, and auto-approval is strongly discouraged.
* There are multiple options for review - do it there and then, or store the plan as an artefact and share with others for review.
* It doesn't matter when you run a stored plan - the plan is the set of changes that will be applied regardless of current state.
* The summary makes very clear if anything is going to be destroyed in bright red text.
Ansible offers some visibility of what it will do with dry runs, although it's not as complete - there's no way to guarantee it will do the same thing next time if changes have been made in the interim.
set to 0 to delete them
1. commit a change that sets to 0
2. deploy change.
3. Make a new commit removing the step.
In Terraform it's just
1. Commit a change removing the resource.
2. Deploy
No cleanups that might be forgotten. No double commits for something that should just be 1.
Terraform doesn't "lose track" of an identifier. It's state file management is pretty robust and it has built in locking for the shared use case.
The Reconciler Pattern basically means:
* there is some notion of "expected" state, which is what you define (declaritively) in the configuration
* there is some "actual" state, which is basically what is running at whatever cloud service, etc. you are dealing with.
* the reconciler's job is to query the actual state, compare it to the expected state, calculate the difference (usually in terms of a graph), then make whatever changes it needs to to bring "actual" in line with "expected".
Kubernetes, SaltStack, and others implement the same pattern (just on different levels of resources) and it's becoming increasingly common and important to understand if you're working with cloud stuff.
https://www.oreilly.com/library/view/cloud-native-infrastruc...
Terraform can only provision cloud resources on AWS/GCP/Azure/other. Usually it gets support first for new products they release. Terraform is very static (see issues with sharing the state file) so it's more indicated to configure very static stuff, like networking and subnets.
My advice to the OP, as it's all new to you, is to learn Ansible. It will require more work than Terraform, but Ansible can be made to perform the same functions as Terraform, and a heck of a lot more stuff that will prove useful to you, if you're looking into how to provision cloud instances.
That makes Ansible sound like it's hard work, but it's actually quite the opposite. It's surprisingly easy to do something useful with it.
Ansible is probably best described as a scripting environment / DSL combination to help you control multiple machines remotely for build and provisioning purposes.
You have 2 separate remote machines, and want to install Postgres on both of them? Use Ansible.
Want to install GIT and then pull your project repo onto two machines? Use Ansible.
Perhaps you have 3 machines, want git on all 3, but Postgres on only one of them? Ansible again.
Where there is an overlap with Terraform, is that Ansible can also be used as an interface to control AWS/Google cloud/Whatever services, which is Terraform's sole purpose. Terraform provides a cloud platform agnostic interface to allow you to spin up new cloud instances, and perform some provisioning tasks, but it will only skim the surface of what you can do with dedicated Ansible scripts.
Ansible is an imperative way of setting up your cloud. You tell it to do certain things, install this package, copy this over there.
Hope it helps
by default you use Ansible modules/roles and specify the desired state.
E.g. have these packages installed, have these directories created/deleted.
Use cases not covered by modules can fallback to using shell commands
If I could add to your answer:
Not only is ansible declarative and idempotent but it also includes countless cloud provisioning modules to bring idempotency to cloud environments.
Terraform may be easier for basic provisioning of cloud resources but I always switch back to ansible cloud modules when I need to do anything complex. Ansible also has the added benefit of easily context switching over to configuring the compute resources after they have launched.
So I would rather say that Ansible is much less declarative than Terraform, because Ansible tasks (the different steps of an Ansible Playbook) are executed sequentially.
The tasks of Ansible are its statements, so yeah we would say that each Ansible task is declarative. And still, a requirement for that would be for the task to use a module/role which is idempotent, right? Another proof, Ansible natively offers loop, blocks, and conditional to control the execution flow throughout its tasks.
(This is not a critic of Ansible. I am happy to use it as is, as a high-level scripting mechanism.)
You write the state you would like resources to be in. When you run terraform plan, it tells you how the state might end up. You run terraform apply and then get to find out what actually happens.
Will it create the resources how I expect? What will the resource's properties be? Will it fail half way into the changes and stop in a broken state? Will it blow away changes without asking/showing me first? Will it refuse to do anything? Who knows.
It's a guessing game. The only way to be sure of what it will or won't do is to write procedural code and tests, so at least you know what decisions it will and won't make.
'Declarative' is just us fooling ourselves that we can make complexity easy to deal with.
To a large degree expressing something is a "poor choice" is an opinion, maybe expert, about optimizations, not about capabilities.
When one is learning, adopting the value judgements of experts is a form of premature optimization that actually prevents learning.
The only way to build your own opinions is through your own experience. You will need to have your own problems, and solve them using a variety of tools, to build your own opinions.
Try both tools in real problems, and the mental model that accrues in your experience will start to guide your opinions about ways to optimize your work.
Also- everybody is just making it up. And all tools suck.
I don’t know Ansible much, but I believe it’s more of a procedure-oriented system, where you declare the steps necessary to reach A, then again to go from A to B. This can be an issue if any item is actually not in the state you expected.
Terraform tells how your Infastructure should look like. Ansible what software should be on your infrastructure/servers.
I tend to use Terraform to describe how the underlying Cloud infrastructure should look like. I use Ansible to describe and configure what software should be running on those servers.
Usage cases:
Simply put Terraform cloud infrastructure provisioning. Ansible server software and configuration files provisioning.
Ansible (mostly) does not refuse to do anything just because the state changed. If you need to make sure something happens, you can be more confident Ansible will do it, because it doesn't care what the state was before now.
Otherwise both are quite good.
> --check
If you're familiar with Packer, then Packer is responsible for creating identical VM images which can be integrated to a CI pipeline and provisioned and baked using Ansible. This baked image is then deployed using Terraform.
Be advised that provisioning in Terraform during VM deployment is not recommended since it increases startup time of the machine. To perform ad hoc configuration management, you use Ansible.
You could very well use Ansible for managing and deploying cloud resources, but that's not what it's meant to do. Moreover, Ansible does not support the concept of state as does Terraform.
Usually Ansible is used for declaring the desired state of the individual servers for example you may use it to manage installed packages and configuration files on the servers.
Whereas with Terraform you declare the desired state of cloud resources for example you may ask Terraform to give you 5 EC2 instances, 1 RDS instance for DB and 1 S3 bucket for storage.
There's some overlap between them but what I've said is largely accurate.
Ansible is much more of an imperative system, sort of "executable YAML". You define a series of tasks in a YAML file. There are predefined tasks for standard things that you need to do when configuring a system, e.g. creating a directory or generating a config file by merging Ansible configuration variables with template. You can and should make these tasks idempotent, but as the system gets more complex, it becomes difficult and runtime can be slow as it compares tasks one by one to the running system.
Both systems suffer somewhat from difficulty in writing code. The fundamental task is to transform configuration variables and templates into running resources. To do that, you need loops, if/then/else logic, etc. Ansible has some constructs, but it is basically string manipulation, with a backdoor of being able to write modules in python. Terraform has a better syntax to define resources. Logic is generally things like ternary operator and list comprehensions. Terraform 0.12 improved this tremendously, but it is still somewhat weak. Ansible has a bit better management of config variables. Terraform tends to make you serialize things through environment vars, and it's awkward to define structure sometimes. Both would benefit greatly from first class functions and programming logic, even as they are "functional", just transforming data.
I love them both, and I hate them both. Terraform is best for provisioning complex infrastructure. Ansible is great for setting up instances, and it's easy for everyone to understand, dev and ops. Here is a complete example of deploying a complex, full-featured app to AWS using Terraform and Ansible: https://github.com/cogini/multi-env-deploy
I feel like we are suffering through a period where the tools are immature. People are focusing on syntax, but we are missing fundamental parts of the way the system should work. https://www.cogini.com/blog/is-it-time-for-lisp-in-devops/
The exact same thing is going on in the Kubernetes world. Back in the .com days, we would laugh at the "HTML programmers", but now we are "YAML programmers".
There are a couple of fundamental ways of managing the new cloud systems, all of which are better or worse depending on what you are doing. There are declarative systems like Terraform or CloudFormation. There is imperative with tasks, like Ansible. There are things that talk directly to the API like boto. There are tools like Pulumi which take a library approach in a general purpose programming language. Dockerfiles are crying out for higher level solutions, which are being developed. Ultimately I like the approach of a dedicated syntax like Terraform, but with more programming capability, or Pulumi.
Am noob. Have done a wee bit of CloudFormation, Docker, k8s. And once completed a Terraform howto. I've never touched Ansible, Chef, Puppet, etc.
I'd love a feature comparison matrix. Or maybe a decision flowchart on how to choose which tool for which job.
--
Update: This comparison was linked upthread. It's pretty good.
https://blog.gruntwork.io/why-we-use-terraform-and-not-chef-...