AWS Advent 2014 – Managing EC2 Security Groups using Puppet
Today’s post on managing EC2 Security Groups with Puppet comes to use from Gareth Rushgrove, the awesome curator of DevOps Weekly and who is currently an engineer at PuppetLabs.
At Puppet Labs we recently shipped a module to make managing AWS easier. This tutorial shows how it can be used to manage your security groups. EC2 Security groups act as a virtual firewall and are used to isolate instances and other AWS resources from each other and the internet.
An example
You can find the full details about installation and configuration for the module in the official READMEbut the basic version, assuming a working Puppet and Ruby setup, is:
|
1 2 3 |
gem install aws-sdk-core puppet module install puppetlabs-aws |
You’ll also want to have your AWS API credentials in environment variables (or use IAM if you’re running from within AWS).
|
1 2 3 |
export AWS_ACCESS_KEY_ID=your_access_key_id export AWS_SECRET_ACCESS_KEY=your_secret_access_key |
First lets create a simple security group called test-sg in the us-east-1 region. Save the following to a file called securitygroup.pp:
|
1 2 3 4 5 6 7 8 9 10 11 |
<span class="pl-st">ec2_securitygroup</span> { <span class="pl-en">'test-sg'</span>: <span class="pl-c1">region</span> => <span class="pl-s1"><span class="pl-pds">'</span>us-east-1<span class="pl-pds">'</span></span>, <span class="pl-c1">ensure</span> => present, <span class="pl-c1">description</span> => <span class="pl-s1"><span class="pl-pds">'</span>Security group for aws advent<span class="pl-pds">'</span></span>, <span class="pl-c1">ingress</span> => [{ <span class="pl-c1">security_group</span> => <span class="pl-s1"><span class="pl-pds">'</span>test-sg<span class="pl-pds">'</span></span>, }], <span class="pl-c1">tags</span> => { <span class="pl-c1">reason</span> => <span class="pl-s1"><span class="pl-pds">'</span>awsadvent<span class="pl-pds">'</span></span>, }, } |
Now lets run Puppet to create the group:
|
1 2 |
puppet apply securitygroup.pp --test |
You should see something like the following output:
|
1 2 3 4 5 6 7 8 |
Info: Loading facts Notice: Compiled catalog for pro.local in environment production in 0.05 seconds Info: Applying configuration version '1418659587' Info: Checking if security group test-sg exists in region us-east-1 Info: Creating security group test-sg in region us-east-1 Notice: /Stage[main]/Main/Ec2_securitygroup[test-sg]/ensure: created Notice: Finished catalog run in 15.22 seconds |
We’re running here with apply and the --test flag so we can easily see what’s happening, but if you have a Puppet master setup you can run with an agent too.
You will probably change your security groups over time as you’re infrastructure evolves. And managing that evolution is where Puppet’s declarative approach really shines. You can have confidence in the description of your infrastructure in code because Puppet can tell you about any changes when it runs.
Next lets add a new ingress rule to our existing group. Modify the securitygroup.pp file like so:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
<span class="pl-st">ec2_securitygroup</span> { <span class="pl-en">'test-sg'</span>: <span class="pl-c1">ensure</span> => present, <span class="pl-c1">region</span> => <span class="pl-s1"><span class="pl-pds">'</span>us-east-1<span class="pl-pds">'</span></span>, <span class="pl-c1">description</span> => <span class="pl-s1"><span class="pl-pds">'</span>Security group for aws advent<span class="pl-pds">'</span></span>, <span class="pl-c1">ingress</span> => [{ <span class="pl-c1">protocol</span> => <span class="pl-s1"><span class="pl-pds">'</span>tcp<span class="pl-pds">'</span></span>, <span class="pl-c1">port</span> => 80, <span class="pl-c1">cidr</span> => <span class="pl-s1"><span class="pl-pds">'</span>0.0.0.0/0<span class="pl-pds">'</span></span>, },{ <span class="pl-c1">security_group</span> => <span class="pl-s1"><span class="pl-pds">'</span>test-sg<span class="pl-pds">'</span></span>, }], <span class="pl-c1">tags</span> => { <span class="pl-c1">reason</span> => <span class="pl-s1"><span class="pl-pds">'</span>awsadvent<span class="pl-pds">'</span></span>, }, } |
And again lets run Puppet to modify the group:
|
1 2 |
puppet apply securitygroup.pp --test |
You should see something like the following output:
|
1 2 3 4 5 6 7 |
Info: Loading facts Notice: Compiled catalog for pro.local in environment production in 0.04 seconds Info: Applying configuration version '1418659692' Info: Checking if security group test-sg exists in region us-east-1 Notice: /Stage[main]/Main/Ec2_securitygroup[test-sg]/ingress: ingress changed [{'security_group' => 'test-sg'}] to '{"protocol"=>"tcp", "port"=>"80", "cidr"=>"0.0.0.0/0"} {"security_group"=>"test-sg"}' Notice: Finished catalog run in 13.59 seconds |
Note the information about changes to the ingress rules as we expected. You can also check the changes in the AWS console.
The module also has full support for the Puppet resource command, so all of the functionality is available from the command line as well as the DSL. As an example lets clean-up and delete the group created above.
|
1 2 |
puppet resource ec2_securitygroup test-sg ensure=absent region=us-east-1 |
Hopefully that’s given you an idea of what’s possible with the Puppet AWS module. You can see more examples of the module in action in the main repository.
Advantages
Some of the advantages of using Puppet for managing AWS resources are:
- The familiar DSL – if you’re already using Puppet the syntax will already be familiar, if you’re not already using Puppet you’ll find lots of good references and documentation
- Puppet is a declarative tool – Puppet is used to declare the desired state of the world, this means it’s useful for maintaining state and changing resources over time, as well as creating new groups
- Existing tool support – whether it’s the Geppetto IDE, testing tools like rspec-puppet or syntax highlighting for your favourite editor lots of supporting tooling already exists
The future
The current preview release of the module supports EC2 instances, security groups and ELB load balancers, with work on support for VPC, Route53 and Autoscaling Groups available soon. We’re looking for as much feedback as possible at the moment so feel free to report issues on GitHub), ask questions on the puppet-user mailing list or contact me on twitter at @garethr