In November of 2018, AWS released S3 Block Public Access, as a method to apply an overarching policy to prevent public access to S3 buckets. The policy contains four options, and can be applied individually, or as a set which provides expected flexibility from an AWS feature (and the excess rope to cause trouble).

When I started consolidating my accounts, and planning out their use, I knew that I would use S3 not only for log aggregation from CloudTrail, but potentially for static web hosting (like this blog). By carefully considering the options in the new feature, I decided that I would apply all of them across all of the accounts at the root of S3. This ensured that no S3 bucket would be exposed unintentionally from any of the accounts.

Armen, if this was so easy, then why are you writing a post about it?

Because it wasn’t!

Oh come on! I’m sure you could whip up a CloudFormation template to apply the policies at the root level of the S3 service… No, you can’t. You can only apply it to a bucket target…

Okay, okay. Well, what about a lambda backed CloudFormation? That should be easy, right?

Nope. The Python 3.7 runtime for Lambda includes v1.9.42 of boto3 which doesn’t include S3Control. The earliest it was made available was in v1.9.46.

Taking this all in, I realized it would require building, and packaging a Lambda with its dependencies, to simply toggle four Booleans.

Let me know when you’re done digesting that fact. I’ll wait. Hah!

I took to my friendly neighborhood GitHub search engine to see if anyone else was as crazy as I was to take this on. I could not believe it when I found this repository by Sander Knape from the Netherlands. Shortly after the AWS announcement, he developed exactly what I was looking for!

The package was developed in Python, using a Makefile as the driver to deploy the CloudFormation. After reviewing the code, I realized that I would need to make changes to support my use of AWS profiles to deploy the policy across the accounts. I also wanted to add a feature to update any of the four Booleans in the policy on a specific account. With those two requirements set, I forked the repository, and started developing.

After making 31 commits over a period of four months, I had built exactly what I was looking for.

The code for the lambda to toggle each of the policy changes was the easiest. The segment below is the meat of the Lambda. There are two functions; one for handling create and update events from the CloudFormation service, and another for the delete event. It took me a while to fully understand how CloudFormation was calling the Lambda, and what it was passing to it at execution time. The piece of AWS documentation that was incredibly helpful (after reading it repeatedly more than I want to admit) was this. Once I understood the process, refining the code to its current state was simple.

As an added note, Sander wrapped his functions with decorators from a helper library to solve the nuances of handling responses and exceptions from CloudFormation during execution of the Lambda. I chose to replace the library he used with the one developed by the AWS open source team, only because of long term support consideration.

@helper.update
@helper.create
def create(event, context):
  client = boto3.client('s3control')
  client.put_public_access_block(
    PublicAccessBlockConfiguration={
      "BlockPublicAcls": bool(event['ResourceProperties']['BlockPublicAcls'] == 'true'),
      "IgnorePublicAcls": bool(event['ResourceProperties']['IgnorePublicAcls'] == 'true'),
      "BlockPublicPolicy": bool(event['ResourceProperties']['BlockPublicPolicy'] == 'true'),
      "RestrictPublicBuckets": bool(event['ResourceProperties']['RestrictPublicBuckets'] == 'true')
    },
    AccountId=event['ResourceProperties']['AccountId']
  )

@helper.delete
def delete(event, context):
  client = boto3.client('s3control')
  client.put_public_access_block(
    PublicAccessBlockConfiguration={
      "BlockPublicAcls": not bool(event['ResourceProperties']['BlockPublicAcls'] == 'true'),
      "IgnorePublicAcls": not bool(event['ResourceProperties']['IgnorePublicAcls'] == 'true'),
      "BlockPublicPolicy": not bool(event['ResourceProperties']['BlockPublicPolicy'] == 'true'),
      "RestrictPublicBuckets": not bool(event['ResourceProperties']['RestrictPublicBuckets'] == 'true')
    },
    AccountId=event['ResourceProperties']['AccountId']
  )

As I mentioned above, the code to make changes to the toggles was simple. Once I figured out how to pass the AccountId from the CloudFormation template to the lambda, setting the values per service to each account programatically from the command line was straight forward. You’ll notice that in each put_public_access_block call in the excerpt above, the second argument is pulling the AccountId from the ResourceProperties array into the event object.

Running the build, and the deploy via make looks like the following:

(s3control) [21:01]$ make -e S3_BUCKET=redacted -e AWS_DEFAULT_PROFILE=security-admin -e S3_PREFIX=lambdas -e S3_CONTROL_ROLE_PATH=/security/ deploy
rm -rf pkg/publicbuckets.py templates/packaged.yaml
Copy lambda source to package staging location...
Attempting to compile cloudformation from template...
aws --profile security-admin cloudformation package \
--template-file templates/template.yaml \
--output-template-file templates/packaged.yaml \
--s3-bucket redacated \
--s3-prefix lambdas
Uploading to lambdas/redacated  9056433 / 9056433.0  (100.00%)
Successfully packaged artifacts and wrote output template to file templates/packaged.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/akale/repository/code/github/lambdas/aws_account_security/block-s3-buckets-cloudformation-custom-resource/templates/packaged.yaml --stack-name <YOUR STACK NAME>
Attempting to deploy resources for custom lambda...

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - security-lambda-s3control
Sleeping for 60 seconds to ensure that IAM role is available to the lambda service...
Attempting to execute custom lambda...

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - security-s3control
Waiting for security-s3control to be complete...
Applying termination protection...
{
    "StackId": "arn:aws:cloudformation:us-east-1:redacated:stack/security-s3control/redacated"
}
{
    "StackId": "arn:aws:cloudformation:us-east-1:redacated:stack/security-lambda-s3control/redacated"
}

The build process packages up the lambda (and the dependencies), uploads the package to the target S3 bucket, and then executes the two CloudFormation stacks.

Two stacks? Why would you have two stacks? Well, the first one sets up the role for use by the lambda, and the lambda function for use by the second execution template.

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: >-
  This template effectively defines the lambda to be used by in applying the
  public S3 blocking permissions.

Parameters:
  S3ControlPolicyName:
    Type: String
  S3ControlRoleName:
    Type: String
  S3ControlRolePath:
    Type: String
  CustomFunctionOutputKeyName:
    Type: String

Resources:
  RoleS3Control:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Ref S3ControlRoleName
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          -
            Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: !Ref S3ControlRolePath
      Policies:
        -
          PolicyName: !Ref S3ControlPolicyName
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              -
                Effect: Allow
                Action:
                  - s3:GetAccountPublicAccessBlock
                  - s3:PutAccountPublicAccessBlock
                Resource: "*"
              -
                Effect: Allow
                Action: kms:Decrypt
                Resource: !Sub 'arn:aws:kms:${AWS::Region}:${AWS::AccountId}:alias/aws/lambda'

      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

  BlockPublicS3BucketsFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ../pkg/
      Handler: publicbuckets.handler
      Runtime: python3.7
      Timeout: 30
      Role: !GetAtt RoleS3Control.Arn
    DependsOn: RoleS3Control

Outputs:
  BlockPublicS3BucketsFunction:
    Value: !GetAtt BlockPublicS3BucketsFunction.Arn
    Export:
      Name: !Ref CustomFunctionOutputKeyName

The execution template maintains the Booleans for each of the toggles in the S3 control policy, and effectively passes them to the lambda.

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  This template will use the lambda custom function to apply the S3 public
  bucket controls to an account.

Parameters:
  CustomFunctionOutputKeyName:
    Type: String
  BPA:
    Type: String
    AllowedValues:
      - True
      - False
  IPA:
    Type: String
    AllowedValues:
      - True
      - False
  BPP:
    Type: String
    AllowedValues:
      - True
      - False
  RPB:
    Type: String
    AllowedValues:
      - True
      - False

Resources:
  BlockPublicS3Buckets:
    Type: Custom::BlockPublicS3Buckets
    Properties:
      ServiceToken:
        Fn::ImportValue: !Ref CustomFunctionOutputKeyName
      AccountId: !Ref AWS::AccountId
      BlockPublicAcls: !Ref BPA
      IgnorePublicAcls: !Ref IPA
      BlockPublicPolicy: !Ref BPP
      RestrictPublicBuckets: !Ref RPB

With the package deployed, the output from s3control get-public-access-block shows that the four policies are set to true.

(s3control) [14:47]$ aws --profile redacted s3control get-public-access-block --account-id redacted
{
    "PublicAccessBlockConfiguration": {
        "BlockPublicAcls": true,
        "IgnorePublicAcls": true,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    }
}

I can then update the policy using the update-policy target in the Makefile.

(s3control) [15:16]$ make -e S3_BUCKET=redacted-security -e AWS_DEFAULT_PROFILE=redacted -e S3_PREFIX=lambdas -e S3_CONTROL_ROLE_PATH=/security/ \
> -e BLOCK_PUBLIC_ACLS=false \
> -e IGNORE_PUBLIC_ACLS=false \
> update-policy
Attempting to update S3 public bucket policy...

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - security-s3control
Waiting for security-s3control to be complete...
(s3control) [15:18]$

If I show the public-access-block again, I can see that the policy values were modified accordingly.

(s3control) [15:18]$ aws --profile redacted s3control get-public-access-block --account-id redacted
{
    "PublicAccessBlockConfiguration": {
        "BlockPublicAcls": false,
        "IgnorePublicAcls": false,
        "BlockPublicPolicy": true,
        "RestrictPublicBuckets": true
    }
}

Now that the configuration is stored in CloudFormation, we can view its state as part of the deployed stack, apply drift detection to the stack, audit against it using a CloudWatch alarm if it does deviate, and correct it if desired. In a larger organization, defining a Service Control Policy to prevent the service level policies to be modified by a poweruser would be the next step.

This is a great example of the flexibilty in applying preventive, detective, and corrective controls, especially with a potentially high risk service like S3. If you’re interested, the GitHub repository for the solution is here.