Creating AMI images with Packer

Hashicorp has a large suite of open source excellent utilities and programs that work with a variety of cloud providers and local system management tools.

In particular I have been working with creating AMI EC2 templates and the packer tools for creating this has been a welcome relief in simplicity and reliability.

The goal is to take a Ubuntu EC2 base image and bake in all the desired utilities and setup so that this image can be used as an AMI in a Launch Configuration for an AutoScaling group that will do the following

• Run the application in a stateless manner (application code does this)
• Accept AWS CodeDeploy updates by running the AWS CodeDeploy agent
• Log to CloudWatch in a defined cloud watch target for centralized logging and searching.  Allows us to create SQS triggers for lambda events and notifications as well as removes the tedium of having to ssh to the particular machine and with an AutoScaling spot instance this request could be changing quite often.

Sample configuration packer file for creating an EC2 AMI image template

{
  "variables": {
    "aws_access_key": "",
    "aws_secret_key": ""
  },
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "{{user `aws_access_key`}}",
    "secret_key": "{{user `aws_secret_key`}}",
    "region": "<your_region>",
    "source_ami": "<ubuntu_EC2_base_image>",
    "instance_type": "<t, m, r, c, whatever series type>",
    "ssh_username": "ubuntu",
    "ami_name": "<asg_scaling_group_name>-{{isotime \"2006-01-02_1504\"}}",
	"ebs_optimized": true,
    "snapshot_tags": {
      "Name": "<asg_scaling_group_name>-{{isotime \"2006-01-02_1504\"}}"
    },
    "iam_instance_profile": "iam_role"
  }],
  "provisioners": [
    {
	  "type": "shell",
      "script": "script1.sh"
    },
    {
      "type":"shell",
      "script": "script2.sh"
    }
  ]
}

In the above I would recommend that you include again the installation of the following through plain bash scripts

• AWS CLI tools
• AWS CodeDeploy agent
• AWS CloudWatch Agent along with the configuration for the target group (sample configuration below that will log a specified local machine log output path to a particular cloud_watch_group)

[general]
state_file = /var/awslogs/state/agent-state

[cloud_watch_group]
datetime_format = %Y-%m-%d %H:%M:%S
file =  <local_machine_log_output_path>
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = cloud_watch_group

• Any other scripts for setup such as python3, etc

Packer command to create AMI

packer build -var 'aws_access_key={"ACCESS_ID"}' -var 'aws_secret_key={"ACCESS_KEY"}' path_to_packer_template.json

At the conclusion of running the packer template should produce a baked and ready AMI