Using MFA with AWS using Python and boto3

Alright, so in the journey of figuring things out the hard way, we’ll just get right into it. I’ve been working on finding a way in AWS to identify which service/resource instances are using what roles, so that we can start eliminating unused roles. AWS does a decent job of providing information, but there’s still a bit of a guessing game. So to solve this, I’ve started writing a little program in python3 that can hunt through the roles and identify which roles are using the roles, if ever. In addition to this, my company has MFA activated, so connecting to AWS requires the right tokens to even start making queries about roles and whatever else.

That said, let’s just get right into how to get the AWS SDK boto3 to connect to AWS using MFA with a virtual device, like Google Authenticator, or Twilio’s Authy app. This took me a bit to figure out because I’m a noob, so I’m writing this down in hopes of it helping other noobs out too.

The Python SDK to connect to AWS is called boto3. This has everything you'd need to connect and use AWS services from the CLI level, but using Python. We'll start from the very beginning assuming you've never used any of this stuff before.

1. Project Setup

First you’ll want to install the AWS-CLI. You can find instructions here.

Once you’ve verified it’s installed on your system, you’ll need a to get a few things in order. First you’ll need an AWS user account (for the love of code, don’t use the AWS root account), because you’re going to need the access key and secret key from that user account.

You’ll then want to make sure you have your MFA serial information so that boto3 can use it when you submit the 6 digit number generated by your virtual MFA device. To set it up and get the info, check out this page.

Next we need to configure the AWS CLI to your user account, so open up your favorite terminal (I’ll use Powershell in Windows, or iTerm2 on mac) and type:

aws configure

This will get you into configuration mode where you can type in those keys you got from your AWS user account. It will ask you for input one at a time, and end up looking something like this:

AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json

What this is actually doing is putting this information into a file that your machine can reference when we get into boto3. This is located in (on Mac) /Users/<your username>/.aws/credentials. You'll also see /Users/<your username>/.aws/config in the same folder location. This credentials file is an ASCII file formatted in INI format, and will look like:

[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
region = us-west-2

This file is what boto3 taps into when it's doing it's thing, so setting this up correctly is important. You will also need to add the MFA information, but this will need to be added to that other file, config. So open that up and you'll want it to look like this:

[default]
mfa_serial = arn:aws:iam::<your AWS account #>:mfa/<your username>

Now that we have both the AWS credentials file and config file set up, we can get into the Python code.

2. Python project setup

I’m using PyCharm because it’s pretty great for Python dev, though VS Code is cool too. In pycharm, set up a new project, I’ve named mine boto3AWS-MFAtutorial, and I'm using Python 3.8.2 as of the time of this article being published.

Click Create and you'll see the project open up like this:

First off delete all the text in main, we don’t need it. We’ll need to get all the packages to make this work. If you’ve got python installed correctly on your machine, you should have access to pip, which can install the packages we need on this project. Open up the terminal at the bottom of PyCharm and type:

pip install boto3

3. Project code

Now we have the AWS SDK made for Python. We’ll need to access it in our program, so the first line needs to import it with:

import boto3

It’ll be grey until we actually use boto3 in our code.

For our tutorial app, we’re going to get the user list from AWS, and with MFA activated, we won’t be able to do so, until we get our ducks in a row, this means we have to have the app log in using our user tokens and create a session with AWS, then use that session to authenticate the MFA, and give us NEW tokens that include the MFA activation.

First step, we’ll need to create our session:

import boto3
session = boto3.Session()

Here boto3 reaches out to those credential and config files we set up earlier, and uses our keys to get us logged in to AWS, and create a session, though MFA isn't validated yet. If we go into debug mode, we can actually drill down into the components of it and find that it has the mfa_serial we added earlier to the config file. We can tap into that by adding another line of code and creating a variable to store that serial number, because we're going to need it to get the MFA validated tokens.

import boto3
session = boto3.Session()
mfa_serial = session._session.full_config['profiles']['default']['mfa_serial']

What this is doing is accessing different sections of nested dictionaries, until we finally get to mfa_serial. Next we need a way to capture the 6 digit value our virtual MFA device is telling us at the moment, luckily python makes this easy, with the function called input().

import boto3
session = boto3.Session()
mfa_serial = session._session.full_config['profiles']['default']['mfa_serial']
mfa_token = input('Please enter your 6 digit MFA code:')

The reason we’re capturing both the mfa_serial and the mfa_token is because we're going to need to use these values to make a call to AWS and get our MFA-validated tokens. The AWS Service that allows us to validate MFA, is AWS Security Token Service, also known as STS. We'll need to create a client of STS in our app to make the call and validate MFA, and get our new tokens.

import boto3
session = boto3.Session()
mfa_serial = session._session.full_config['profiles']['default']['mfa_serial']
mfa_token = input('Please enter your 6 digit MFA code:')
sts = session.client('sts')
MFA_validated_token = sts.get_session_token(SerialNumber=mfa_serial, TokenCode=mfa_token)

So first we’ve got sts = session.client('sts') which creates a client object for us to make calls to the AWS API, then we use that to get MFA_validated_token from STS. From the STS client, we're using the call get_session_token(). Notice the arguments being put into the method there. These are known as keyword arguments, and when you see python methods having an input like **kwargs, this is what it looks like to use them. From the boto3 documentation on STS clients, we can see that they use get_session_token(**kwargs) as the input. Since this was my first time seeing this (I'm pretty green to python) I had to figure out what **kwargs even meant, and how to use them.

In a nutshell, it means you have to define a keyword, and it’s value, and the double asterisks mean that there can be any number of these keyword args. So in our example here, the boto3 docs define the request syntax as:

response = client.get_session_token(
    DurationSeconds=123,
    SerialNumber='string',
    TokenCode='string'
)

This gives us our keywords (AKA parameters/arguments), and the type of data that they accept. Because there are **, you can use all or none of these, though it wouldn’t be much help to use none of them, we want to make the call using the data we captured earlier using mfa_serial and mfa_token.

MFA_validated_token = sts.get_session_token(SerialNumber=mfa_serial, TokenCode=mfa_token)

Therefore we make the call using both of them , and we’re calling our response MFA_validated_token, which is a JSON object containing the following dictionaries:

{
    'Credentials': {
        'AccessKeyId': 'string',
        'SecretAccessKey': 'string',
        'SessionToken': 'string',
        'Expiration': datetime(2015, 1, 1)
    }
}

Now we have our MFA validated credentials, so we can now make calls to other resources and services, submitting these credentials, instead of our user credentials which won’t work.

If I want to use IAM to get a list of users, I’ll need to create a client similar to what I did with STS, and then use the list_users(**kwargs) method to get a list of all the AWS users I have permission to see. Let's add all that code in now and explain it after.

import boto3
session = boto3.Session()
mfa_serial = session._session.full_config['profiles']['default']['mfa_serial']
mfa_token = input('Please enter your 6 digit MFA code:')
sts = session.client('sts')
MFA_validated_token = sts.get_session_token(SerialNumber=mfa_serial, TokenCode=mfa_token)
iam = session.client('iam', 
               aws_session_token=MFA_validated_token['Credentials']['SessionToken'],
               aws_secret_access_key=MFA_validated_token['Credentials']['SecretAccessKey'],
               aws_access_key_id=MFA_validated_token['Credentials']['AccessKeyId']
                      )
userList = iam.list_users()

Ok so what’s happening here is we create the IAM client, and use that same keyword argument style parameters to create the client with MFA validation. If we didn’t do this, and MFA is properly set up in AWS, you’ll get an error saying you were explicitly denied, but by putting in the SessionToken, SecretAccessKey, and AccessKeyId we acquire from STS, we're able to successfully complete this query. Then with the client object created, we're able to use list_users(**kwargs) and get back the list of AWS users as a big JSON object of nested dictionaries.

There you have it, if you want to pretty print the result in the console to see what the list actually looks like, we’ll add one more bit of code, using the Python json module. This is native to Python, so we don't need to use PIP to install it.

import boto3
import json <---- this is new up here, don't miss this
session = boto3.Session()
mfa_serial = session._session.full_config['profiles']['default']['mfa_serial']
mfa_token = input('Please enter your 6 digit MFA code:')
sts = session.client('sts')
MFA_validated_token = sts.get_session_token(SerialNumber=mfa_serial, TokenCode=mfa_token)
iam = session.client('iam', 
               aws_session_token=MFA_validated_token['Credentials']['SessionToken'],
               aws_secret_access_key=MFA_validated_token['Credentials']['SecretAccessKey'],
               aws_access_key_id=MFA_validated_token['Credentials']['AccessKeyId']
                      )
userList = iam.list_users()
print(json.dumps(userList , default=str, indent=2))

And there you have it. Run your code, it should ask you for your 6 digit MFA (I’m using Authy for this, but I also use Google Authenticator a lot too), and return the user list from your AWS account in your console. From there…. you can do whatever you want or need with that userList object. The final code looks like this:

Hope this helps folks new to AWS SDK, boto3, and Python. Hit me up in the comments if you have any questions!