Amazon EC2 SSH Public Keys via IAM Service

There will come a day when we can provision users and provide the necessary authorization to required enterprise resources with easy, but until that day comes there will be hacks like this one.

This is very much Proof of Concept (PoC), I have not tested this beyond my network, and there would be a lot of angles to consider with any type of deployment, security (sanitization and additional logic), IAM SSH Public Keys were designed for CodeCommit, and other unknowns.

My first attempt was to create a PAM module that would authenticate SSH users via aws-sdk-cpp libraries, accessing the API using instance profile / EC2 IAM roles, allowing EC2 instances to obtain credentials to access the information. This seemed like a good route, but trying to hack a PAM module together that would be portable enough for distribution would be a challenge, static versions of aws-sdk-cpp, etc. I will revisit this in the future, but for now, I will move on.

On my second attempt, I create a Bash script that would produce the correct output for SSH AuthorizedKeysCommand directive using SSH public keys from AWS IAM service via awscli. As the EC2 instance can utilize the instance profile role via aws command. This didn’t come without headaches, permission issues, etc.

Check sshd DEBUG logs if having issues with execution.

chmod 0744 /usr/local/bin/aws-iam-authorized-keys
chown root:root /usr/local/bin/aws-iam-authorized-keys

Once I was finally able to get sshd to execute aws-iam-authorized-keys (my aptly named script), it was able to query for the username’s public SSH keys in IAM and output them in the proper format.

For the EC2 instance to access the IAM to list and retrieve keys, it will require an instance profile, that allows ListSSHPublicKeys and GetSSHPublicKey actions, similar to the following IAM role policy I have been using. You can read more in the user guide under IAM Roles for Amazon EC2:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetSSHPublicKey",
                "iam:ListSSHPublicKeys"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

The two /etc/ssh/sshd_config directives

AuthorizedKeysCommand /usr/local/bin/aws-iam-authorized-keys
AuthorizedKeysCommandUser nobody

The daemon sshd will pass the username to the script or application that you supply to AuthorizedKeysCommand as the first argument, and will execute it as the user-defined in AuthorizedKeysCommandUser.

In this scenario, I have a user in IAM with the name jonathan, SSH public key, SSH private key on my notebook (passphrase protected), local EC2 Linux user jonathan. This will allow my notebook SSH client to authenticate, after I provide the passphrase of course.

jonathan$ ssh [email protected]
Enter passphrase for key '/Users/jonathan/.ssh/id_rsa': 
Last login: Thu Jan 26 03:44:03 2017 from

 __| __|_ )
 _| ( / Amazon Linux AMI
 ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2016.09-release-notes/

[[email protected] ~] $ id uid=501(jonathan) gid=501(jonathan) groups=501(jonathan)
[[email protected] ~] $ ls -la .ssh/ total 8 drwxr-xr-x 2 jonathan jonathan 4096 Jan 26 02:12 . drwx------ 3 jonathan jonathan 4096 Jan 26 03:26 .. 
[[email protected] ~]

Eureka! We have made a successful authentication. I have increased the log level to DEBUG using the sshd_config directive LogLevel. With this, we can see what sshd is doing.

sshd[5457]: debug1: userauth-request for user jonathan service ssh-connection method none [preauth]
sshd[5457]: debug1: attempt 0 failures 0 [preauth]
sshd[5457]: debug1: PAM: initializing for "jonathan"
sshd[5457]: debug1: PAM: setting PAM_RHOST to "X"
sshd[5457]: debug1: PAM: setting PAM_TTY to "ssh"
sshd[5457]: debug1: userauth-request for user jonathan service ssh-connection method publickey [preauth]
sshd[5457]: debug1: attempt 1 failures 0 [preauth]
sshd[5457]: debug1: test whether pkalg/pkblob are acceptable [preauth]
sshd[5457]: debug1: temporarily_use_uid: 99/99 (e=0/0)
sshd[5457]: debug1: restore_uid: 0/0
sshd[5457]: debug1: temporarily_use_uid: 99/99 (e=0/0)
sshd[5457]: Found matching RSA key:
sshd[5457]: debug1: restore_uid: 0/0
sshd[5457]: Postponed publickey for jonathan from X port 53067 ssh2 [preauth]
sshd[5457]: debug1: userauth-request for user jonathan service ssh-connection method publickey [preauth]
sshd[5457]: debug1: attempt 2 failures 0 [preauth]
sshd[5457]: debug1: temporarily_use_uid: 99/99 (e=0/0)
sshd[5457]: debug1: restore_uid: 0/0
sshd[5457]: debug1: temporarily_use_uid: 99/99 (e=0/0)
sshd[5457]: Found matching RSA key:
sshd[5457]: debug1: restore_uid: 0/0
sshd[5457]: debug1: ssh_rsa_verify: signature correct
sshd[5457]: debug1: do_pam_account: called
sshd[5457]: Accepted publickey for jonathan from X port 53067 ssh2: RSA
sshd[5457]: debug1: monitor_child_preauth: jonathan has been authenticated by privileged process
sshd[5457]: debug1: monitor_read_log: child log fd closed
sshd[5457]: debug1: SELinux support disabled
sshd[5457]: debug1: PAM: establishing credentials
sshd[5457]: pam_unix(sshd:session): session opened for user jonathan by (uid=0)
sshd[5457]: User child is on pid 5513
sshd[5513]: debug1: PAM: establishing credentials
sshd[5513]: debug1: permanently_set_uid: 501/501
sshd[5513]: debug1: Entering interactive session for SSH2.

We can verify the output by running aws-iam-authorized-keys from the command line on an EC2 instance. Output was reduced for brevity.

jonathan$ /usr/local/bin/aws-iam-authorized-keys jonathan
ssh-rsa AAAAB3Nza...noc7qee3 jonathan

Although I am authenticating to a local Linux user account, which doesn’t scale that well, the account information could have come from an external directory like Active Directory or RADIUS via PAM modules.

To recap the instance-side tool chain is as follows, sshd -> ams-iam-authorized-keys -> awscli -> IAM.

You can find the script in Downloads. Enjoy.

Jan 26th, 2017 • Posted in Amazon Web Service, Bash, CLI, EC2, IAM, Linux, SSH
No comments yet.

Leave a comment

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>