AWS Advent 2014 – Using Terraform to build infrastructure on AWS

Today’s post on using Terraform to build infrastructure on AWS comes from Justin Downing.

Introduction

Building interconnected resources on AWS can be challenging. A simple web application can have a load balancer, application servers, DNS records, and a security group. While a sysadmin can launch and manage these resources from the AWS web console or a CLI tool like fog, doing so can be time consuming and tedious considering all the metadata you have to shuffle amongst the other resources.

An elegant solution to this problem has been solved by the fine folks at Hashicorp: Terraform. This tool aims to take the concept of “infrastructure as code” and add the missing pieces that other provisioning tools like fog miss, namely the glue to interconnect your resources. For anyone with a background in software configuration management (Chef, Puppet), then using Terraform should be a natural fit for describing and configuring infrastructure resources.

Terraform can be used with several different providers including AWS, GCE, and Digital Ocean. We will be discussing provisioning resources on AWS. You can read more about the built-in AWS provider here.

Installation

Terraform is written in go and distributed as a package of binaries. You can download the appropriate package from the website. If you are using OSX and homebrew, you can simply brew install terraform to get everything installed and setup.

Configuration

Now, that you have Terraform installed, let’s build some infrastructure! Terraform configuration files are text files that resemble JSON, but are more readable and can include comments. These files should end in .tf (more details on configuration is available here). Rather than invent an example to use Terraform with AWS, I’m going to step through the example published by Hashicorp.

NOTE: I am assuming here that you have AWS keys capable of creating/terminating resources. Also, it would help if had the AWS CLI is installed and configured as Terraform will use those credentials to interract with AWS. The example below is using AWS region us-west-2.

Let’s use the AWS Two-Tier example to build an ELB and EC2 instance:

Here, we initialized a new directory with the example. Then, we created a new keypair and saved the private key to our directory. Here, you will note the files with the tf extension. These are the configuration files used to describe the resources we want to build. As the name indicates, one is the main configuration, one contains the variables used, and one describes the desired output. When you build this configuration, Terrraform will combine all .tf files in the current directory to greate theresource graph.

Make a Plan

I encourage you to review the configuration details in main.tf, variables.tf, and outputs.tf. With the help of comments and descriptions, it’s very easy to learn how different resources are intended to work together. You can also run plan to see how Terraform intends to build the resources you declared.

This also doubles as a linter by checking the validity of your configuration files. For example, if I comment out the instance_type in main.tf, we receive an error:

Variables

You will note that some pieces of the configuration are parameterized. This is very useful when sharing your Terraform plans, committing them to source control, or protecting sensitive data like access keys. By using variables and setting defaults for some, you allow for better portability when you share your Terraform plan with other members of your team. If you define a variable that does not have a default value, Terraform will require that you provide a value before proceeding. You can either (a) provide the values on the command line or (b) write them to a terraform.tfvars file. This file acts like a “secrets” file with a key/value pair on each line. For example:

Due to the sensitive information included in this file, it is recommended that you includeterraform.tfvars in your source control ignore list (eg: echo terraform.tfvars >> .gitignore) if you want to share your plan.

Build Your Infrastructure

Now, we can build the resources using apply:

The output above is truncated, but Terraform did a few things for us here:

1. Created a ‘terraform-example’ security group allowing SSH and HTTP access
2. Created an EC2 instances from the Ubuntu 12.04 AMI
3. Created an ELB instance and used the EC2 instance as its backend
4. Printed the ELB public DNS address in the Outputs section
5. Saved the state of your infrastructure in a terraform.tfstate file

You should be able to open the ELB public address in a web browser and see “Welcome to Nginx!” (note: this may take a minute or two after initialization in order for the ELB health check to pass).

The terraform.tfstate file is very important as it tracks the status of your resources. As such, if you are sharing your configurations, it is recommended that you include this file in source control. This way, after initializing some resources, another member of your team will not try and re-initialize those same resources. In fact, she can see the status of the resources with terraform show. In the event the state has not been kept up-to-date, you can use terraform refresh to update the state file.

And…that’s it! With a few descriptive text files, Terraform is able to build cooperative resources on AWS in a matter of minutes. You no longer need complicated wrappers around existing AWS libraries/tools to orchestrate the creation or destruction of resources. When you are finished, you can simply run terraform destroy to remove all the resources described in your .tf configuration files.

Conclusion

With Terraform, building infrastructure resources is as simple as describing them in text. Of course, there is a lot more you can do with this tool including managing DNS records and configure Mailgun. You can even mix these providers together in a single plan (eg: EC2 instances, DNSimple records, Atlas metadata) and Terraform will manage it all! Check out the documentation and examples for the details.

Terraform Docs: https://terraform.io/docs/index.html
Terraform Examples: https://github.com/hashicorp/terraform/tree/master/examples