In the first post of this series, I described my thought process (or insanity) of this entire project, and made decent progress on my list of requirements by creating the SQS queue, and ensuring least privilege using CloudFormation. The intent of the project being to receive notification(s) indicating the final status of a CodeBuild build without having to continually poll the CodeBuild API, or view the console. I decided to use SQS instead of a combination of SQS, and SNS to keep complexity of the project down, and runtime costs low (effectively negligible considering the number of builds/month). In this post, I’ll discuss how I came up with the message delivery system from within CodeBuild, and reasoning behind the message templates.

The first step of this process was to figure out how (and when) the message was to be sent. The buildspec file that is included as part of the CodeBuild project is quite robust in its definition. A CodeBuild project has two main phases: install, and build. I knew that I wanted notifications from each of the two phases, regardless of state. After reviewing the buildspec, and testing various implementations, I decided to use the finally block in both phases to make the call to SQS. Here’s an excerpt from the working spec:

phases:
  install:
    runtime-versions:
      ruby: 2.6
    commands:
      - gem install bundler
      - bundle install
    finally:
      - >-
        aws sqs send-message --queue-url $SQS_QUEUE --message-body
        "$(sed -e 's/insert-build-step/install/g' -e s/insert-build-status/${CODEBUILD_BUILD_SUCCEEDING}/g .build/sqs-send_message_body.json)"
        --message-attributes "$(sed -e s/insert-epoch/${CODEBUILD_START_TIME}/g -e s/insert-logpath/${CODEBUILD_LOG_PATH}/g .build/sqs-send_message_attributes.json)"
  build:
    commands:
      - bundle exec jekyll build
    finally:
      - >-
        aws sqs send-message --queue-url $SQS_QUEUE --message-body
        "$(sed -e 's/insert-build-step/build/g' -e s/insert-build-status/${CODEBUILD_BUILD_SUCCEEDING}/g .build/sqs-send_message_body.json)"
        --message-attributes "$(sed -e s/insert-epoch/${CODEBUILD_START_TIME}/g -e s/insert-logpath/${CODEBUILD_LOG_PATH}/g .build/sqs-send_message_attributes.json)"

The two most interesting pieces of this code are the use of the $SQS_QUEUE environment variable, and the fact that I am calling the AWS CLI directly within the build without having to declare where the calling credentials are going to come from.

The environment variable was easy, as you can define any number of variables at the top of the buildspec:

env:
  variables:
    JEKYLL_ENV: 'production'
    SQS_QUEUE: 'https://sqs.us-east-1.amazonaws.com/redacted/redacted'

The second was solved when I provided the allowed action to the role that was being assumed by the CodeBuild build:

    - { CodeBuildLogGroupArn: !Select [0, !Split [':*', !GetAtt CodeBuildLogGroup.Arn ]]}
-
  Effect: Allow

Now that I covered the how, and when, I needed to figure out the what in a way that was dynamic and didn’t require manual intervention before each build. After toying with a couple of idaes, the one that I ended up going to production with was one that you can see in use above. I defined two templates, one for the message body, and another for the message attributes (more detail here about the difference [here][external-link=2]). I’m a fan of sending data payloads in JSON (because it’s MUCH easier to handle than XML), and so I created the following two templates:

{
  "timestamp": {
    "DataType": "Number.Epoch",
    "StringValue": "insert-epoch"
  },
  "logpath": {
    "DataType": "String",
    "StringValue": "insert-logpath"
  }
}
{
  "build-step": {
    "DataType": "String",
    "StringValue": "insert-build-step"
  },
  "build-status": {
    "DataType": "Number.Boolean",
    "StringValue": "insert-build-status"
  }
}

For the message attributes, I wanted two values: the timestamp that the build started, and the path of the CloudWatch log for further diagnosis, and troubleshooting in case the build went sideways. For the message body, the build step, and whether or not it succeeded or failed.

The trick to filling the templates with the appropriate values during runtime was to use sed. I am, and always will be a fan of regular expressions. The wonders that I have conjured over the years with it still amaze me, and I will use it any time that I can. It was an easy decision in this case, and fit into the execution of the solution as expected.

In the end, the entire message delivered by SQS looks like this:

{
    "Messages": [
        {
            "MessageId": "redacted",
            "ReceiptHandle": "redacted",
            "MD5OfBody": "redacted",
            "Body": "{\n  \"build-step\": {\n  \"DataType\": \"String\",\n  \"StringValue\": \"install\"\n  },\n  \"build-status\": {\n  \"DataType\": \"Number.Boolean\",\n  \"StringValue\": \"1\"\n  }\n}",
            "Attributes": {
                "SenderId": "redacted",
                "ApproximateFirstReceiveTimestamp": "1572224773770",
                "ApproximateReceiveCount": "1",
                "SentTimestamp": "1572224772655"
            },
            "MD5OfMessageAttributes": "redacted",
            "MessageAttributes": {
                "logpath": {
                    "StringValue": "redacted",
                    "DataType": "String"
                },
                "timestamp": {
                    "StringValue": "1572224672446",
                    "DataType": "Number.Epoch"
                }
            }
        }
    ]
}

For now, I have a shell loop that runs for a few minutes after I push to the release repository that contains the following:

aws sqs receive-message --queue-url https://sqs.us-east-1.amazonaws.com/redacted/redacted \
--attribute-names All --message-attribute-names All --max-number-of-messages 2 --wait-time-seconds 20

Note: I cut the –attribute-names argument, as I didn’t need it passed my testing phase.

Overall, I was quite pleased at how well this worked out, and no longer fire up the AWS console in my browser after pushing to the release repository to watch the build. Achievement unlocked! ;)