Automate OS Image Build Pipelines with EC2 Image Builder #AWS

#Amazon

Earlier in my career, I can recall being assigned the task of creating and maintaining operating system (OS) images for use by my development team. This was a time-consuming process, sometimes error-prone, needing me to manually re-create and re-snapshot images frequently. As I’m sure you can imagine, it also involved a significant amount of manual testing!

Today, customers still need to keep their images up to date and they do so either by manually updating and snapshotting VMs, or they have teams that build automation scripts to maintain the images, both of which can still be time consuming, resource intensive, and error-prone. I’m excited to announce the availability of EC2 Image Builder, a service that makes it easier and faster to build and maintain secure OS images for Windows Server and Amazon Linux 2, using automated build pipelines. The images created by EC2 Image Builder can be used with Amazon Elastic Compute Cloud (EC2) and on-premises, and can be secured and hardened to help comply with applicable InfoSec regulations. AWS provides security hardening policies that you can use as a starting point to meet the “Security Technical Implementation Guide (STIG)” standard needed to operate in regulated industries.

The pipelines that you can configure for EC2 Image Builder include the image recipe, infrastructure configuration, distribution, and test settings, to produce the resulting images. This includes the ability to automatically provision images as new software updates, including security patches, become available. As new images are created by the pipelines, you can additionally configure automated tests to be run to validate the image, before then distributing it to AWS regions that you specify. EC2 Image Builder can be used with EC2 VM Import/Export to build images in multiple formats for on-premises use, including VMDK, VHDX, and OVF. When testing you can use a combination of AWS-provided tests and custom tests that you have authored yourself.

Let’s take a look at how to get started using EC2 Image Builder.

Creating an OS Image Build Pipeline
From the EC2 Image Builder console homepage I can quickly get started by clicking Create image pipeline. Here, I’m going to construct a pipeline that will build a custom Amazon Linux 2 image. The first step is to define the recipe which involves selecting the source image to start from, the build components to apply to the image being created, and the tests to be run.

Starting with the source image, I’m going to select a managed image provided by EC2 Image Builder. Note that I can also choose other images that either I have created, or that have been shared with me, or specify a custom AMI ID.

Next I select the build components to include in the recipe – in other words, the software I want to be installed onto the new image. From within the wizard I have the option to create a new build component by clicking Create build component. Build components have a name (and optional description), a target operating system, an optional AWS Key Management Service (KMS) key to encrypt the component, and a YAML document that specifies the set of customization steps for the component. Build components can also be versioned, so I have a lot of flexibility in customizing the software to apply to my image. I can create, and select, multiple build components and don’t have to do all my customization from one component.

For this post however I’ve clicked Browse build components and selected some Amazon-provided components for Amazon Corretto, Python 3 and PowerShell Core.

The final step for the recipe is to select tests to be applied to the image to validate it. Just as with build components, I can create and specify tests within the wizard, and I have the same capabilities for defining a test as I do a build component. Again though, I’m going to keep this simple and click Browse tests to select an Amazon-provided test that the image will reboot successfully (note that I can also select multiple tests).

That completes my recipe, so I click Next and start to define my pipeline. First, I give the pipeline a name and also select an AWS Identity and Access Management (IAM) role to associate with the EC2 instance to build the new image. EC2 Image Builder will use this role to create Amazon Elastic Compute Cloud (EC2) instances in my account to perform the customization and testing steps. Pipeline builds can be performed manually, or I can elect to run them on a schedule. I have the flexibility to specify my schedule using simple Day/Week/Month period and time-of-day selectors, or I can use a CRON expression.

I selected a managed IAM policy (EC2InstanceProfileForImageBuilder) with just enough permissions to use common AWS-provided build components and and run tests. When you start to use Image Builder yourself, you will need to set up a role that has enough permissions to perform your customizations, run your tests, and write troubleshooting logs to S3. As a starting point for setting up the proper permissions, I recommend that you attach the AmazonSSMManagedInstanceCore IAM policy to the IAM role attached to the instance.

Finally for the pipeline I can optionally specify some settings for the infrastructure that will be launched on my behalf, for example the size of instance type used when customizing my image, and an Amazon Simple Notification Service (SNS) topic that notifications can be forwarded to. I can also take control of Amazon Virtual Private Cloud related settings should I wish.

If the operating system of the image I am building is associated with a license, I can specify that next (or create a new license configuration on-the-fly), along with a name for my new image and also the AWS regions into which the new image will be shared, either publicly or privately.

Clicking Review, I can review all of my settings and finally click Create Pipeline to complete the process.

Even though when I configured my pipeline I asked for it to run daily at 06:00 hours UTC, I can still run it whenever I wish. Selecting the pipeline, I click Actions and then Run pipeline.

Once the build has completed, the AMI will be ready to launch from the Amazon EC2 console!

Thinking back to my earlier career years and the tasks assigned to me, this would have saved me so much time and effort! EC2 Image Builder is provided at no cost to customers and is available in all commercial AWS Regions. You are charged only for the underlying AWS resources that are used to create, store, and share the images.

— Steve