How to get Cloudformation Events available in your CI/CD pipeline

I want to look at a long-standing problem that still persists in many Platform, DevOps, and Development teams today. 

Deploying infrastructure on AWS as code is a well established practice in many organisations, and for those who aren’t doing it just yet, most are on their way. 

When it comes to deploying infrastructure as code, the tool of choice is very often CloudFormation (AWS’ infrastructure as code tool). The usual route is to put CloudFormation deployments in a pipeline (we engineers love pipelines), and the easiest way to do this is via the ‘aws cloudformation deploy’ command line interface (CLI) command. 

The issue

If you’ve used the AWS CLI before, you will know the problem I’m going to talk about… All you see in your pipeline is a success or failure message regarding the CloudFormation deployment. If it passes, you get this:

If it fails, you get this…

This isn’t very useful at all! It doesn’t tell you about AWS resources successfully deployed, which resources errored and most importantly why they errored. If you want any more information, you can use the CLI to manually query the stack events there. Or you can login to the console to see what’s up. To be able to see the deployment error from the pipeline would help engineers more quickly understand the issue and find a fix, as opposed to wasting time logging into AWS and trawling through the CloudFormation UI to find out what happened. Developer experience is becoming more important in organisations, and allowing developers to have the information they need to make decisions in front of them (over logging into multiple  different systems) is a large part of this. 

I experienced the above issue at a client site recently where they didn’t have any defaults for deploying CloudFormation, and I didn’t want to overcomplicate things so we used the CLI call in our pipeline. There are other CloudFormation calls you can use to deploy changes to CloudFormation stacks via updating the stack directly or using change sets, but these commands have similar problems with getting events. It soon became very obvious that we needed at least some information about the CloudFormation failure in the pipeline. Having to trudge through the AWS user interface is difficult enough, and not having this information available to me in my pipeline made things even more challenging!

So what options do you have to gather more information when your CloudFormation deployment fails?

1. Write your own wrapper script - If you Google this issue, there are a lot of people tackling this same issue out there. You start with getting a list of all stack events after the deployment (like what the failure example above listed), and then doing some sort of parsing while the deployment is happening (or after the deployment has finished depending on your use case). It’s fine, but not ideal. 
   

2. Use an open source tool like StackUp - When chatting about this issue with the Midnyte City crew there was definitely a preference for StackUp, which provides a CLI interface for interacting with CloudFormation. It does much more than the standard AWS CLI, handling the events nicely for you. It even does things like cfn-lint, which helps you validate your cloudformation and pick up errors before deploying your resources. Leveraging open source projects to solve problems like this can be a good quick win.
   

3. Go without events in your pipeline - This is obviously possible, but not recommended. It’s inconvenient having to login to AWS (either via CLI or console) to see what’s happening with a failed deployment. When you see that red on your pipeline, it’s much nicer being able to quickly check what has happened. 
   

Final words 

Only seeing a success or failure message regarding your Cloudformation deployment isn’t by any means a new issue, but it’s one that will keep appearing as more and more organisations bed down their approach to deploying infrastructure and Development teams perform this task. Engineers should be able to see events in their pipelines to quickly fix any issues that may arise. Ideally you can leverage work already done, by using something like StackUp, or reusing any prior art in your organisation. The fan favourite from Midnyte City folks though is StackUp. It’s easy to get up and going, and provides many of the niceties that the AWS CLI just doesn’t. So the next time you are in a team and looking at deploying cloudformation, I suggest you check that out.