In November of 2018, AWS released S3 Block Public Access, as a
method to apply an overarching policy to prevent public access to S3 buckets.
The policy contains four options, and can be applied individually, or as a set
which provides expected flexibility from an AWS feature (and the excess rope to
cause trouble).
When I started consolidating my accounts, and planning out their use, I knew
that I would use S3 not only for log aggregation from
CloudTrail, but potentially for static web hosting (like this blog). By
carefully considering the options in the new feature, I decided that I would
apply all of them across all of the accounts at the root of S3. This ensured
that no S3 bucket would be exposed unintentionally from any of the
accounts.
Armen, if this was so easy, then why are you writing a post about it?
Because it wasn’t!
Oh come on! I’m sure you could whip up a CloudFormation template to apply the
policies at the root level of the S3 service… No, you can’t.
You can only apply it to a bucket target…
Okay, okay. Well, what about a lambda backed CloudFormation? That should be
easy, right?
Nope. The Python 3.7 runtime for Lambda includes v1.9.42 of
boto3 which doesn’t include S3Control. The earliest it was made available was in
v1.9.46.
Taking this all in, I realized it would require building, and packaging a
Lambda with its dependencies, to simply toggle four Booleans.
Let me know when you’re done digesting that fact. I’ll wait. Hah!
I took to my friendly neighborhood GitHub search engine to see if anyone else
was as crazy as I was to take this on. I could not believe it when I found
this repository by Sander Knape from the
Netherlands. Shortly after the AWS announcement, he developed exactly what I was
looking for!
The package was developed in Python, using a Makefile as the driver to deploy
the CloudFormation. After reviewing the code, I realized that I would need to
make changes to support my use of AWS profiles to deploy the policy across the
accounts. I also wanted to add a feature to update any of the four Booleans in
the policy on a specific account. With those two requirements set, I forked the
repository, and started developing.
After making 31 commits over a period of four months, I had built exactly
what I was looking for.
The code for the lambda to toggle each of the policy changes was the easiest.
The segment below is the meat of the Lambda. There are two functions; one for
handling create and update events from the CloudFormation service, and another
for the delete event. It took me a while to fully understand how CloudFormation
was calling the Lambda, and what it was passing to it at execution time. The
piece of AWS documentation that was incredibly helpful (after reading it
repeatedly more than I want to admit) was this. Once I
understood the process, refining the code to its current state was simple.
As an added note, Sander wrapped his functions with decorators from a
helper library to solve the nuances of handling responses and exceptions from
CloudFormation during execution of the Lambda. I chose to replace the library he
used with the one developed by the AWS open source team, only
because of long term support consideration.
As I mentioned above, the code to make changes to the toggles was simple. Once I
figured out how to pass the AccountId from the CloudFormation template to the
lambda, setting the values per service to each account programatically from the
command line was straight forward. You’ll notice that in each
put_public_access_block call in the excerpt above, the second argument is
pulling the AccountId from the ResourceProperties array into the event object.
Running the build, and the deploy via make looks like the following:
The build process packages up the lambda (and the dependencies), uploads the package to the target S3 bucket, and then
executes the two CloudFormation stacks.
Two stacks? Why would you have two stacks? Well, the first one sets up the role for use by the lambda, and the lambda
function for use by the second execution template.
The execution template maintains the Booleans for each of the toggles in the S3 control policy, and effectively passes
them to the lambda.
With the package deployed, the output from s3control get-public-access-block
shows that the four policies are set to true.
I can then update the policy using the update-policy target in the Makefile.
If I show the public-access-block again, I can see that the policy values were
modified accordingly.
Now that the configuration is stored in CloudFormation, we can view its state as
part of the deployed stack, apply drift detection to the stack, audit against it using a
CloudWatch alarm if it does deviate, and correct it if desired. In a larger
organization, defining a Service Control Policy to prevent
the service level policies to be modified by a poweruser would be the next step.
This is a great example of the flexibilty in applying preventive, detective, and
corrective controls, especially with a potentially high risk service like S3. If you’re interested, the GitHub
repository for the solution is here.
If you have a thought about this post, share it with me! Tweet to @kriation