AWS Transfer SFTP with CDK and a Custom Host Key

And a few tricks for managing SSH Keys in Secrets Manager

There must be an easier way! Photo by Sara Kurfeß on Unsplash

AWS Transfer is an enterprise-focused product to handle secure file transfers from the public and inside the corporate VPC. Like everything AWS, it is designed for high availability and virtually limitless scaling. Unlike a lot of AWS services, it has a relatively high monthly fixed cost.

This proof-of-concept project came out of the need to deploy and test AWS Transfer in development and pre-production environments — where we can deploy and destroy easily to minimise cost.

One of the key issues to solve was the re-use of the server's private key. By default, the server is created with an AWS-managed key that cannot be exported or shared. This creates some issues with client connections if moving to a new server, as connections will fail until the new key is accepted by the client. In production, it is an issue when operating DR regions, and in development, it’s a pain for the test scripts.

AWS provides a method to import a custom host key to allow for key re-use, targeted at a migration use-case where we are moving users from another SFTP solution to AWS Transfer. In the console, it can only be done when the server is created, but fortunately, it’s also available in the API. Normally that would mean a simple use of CDK Custom Resources to make the API call. In this case, it proved a little more difficult — it seems the API is very picky about the formatting of the key text. My plan to store the SSH private key in Secrets Manager for secure re-use needed a little tweaking.

Here’s how I solved it, along with a basic demonstration of using upload file events to trigger Lambda or email notifications via SNS.

Of course, the full solution is available on GitHub.

Overview

The demo sets up an AWS Transfer Server with:

• Public IP Addresses and optional CIDR address filtering
• Internal VPC connection
• User(s) configured with SSH Key authentication
• Optional custom domain and hostname
• Optional custom server Host Key
• Optional archive bucket and Lambda function to move
• Optional email notifications via SNS on file uploads

Demo Overview

The CDK deployment project also includes a custom Lambda resource to update the Server Host Key with the one you create yourself.

Custom Private SSH Key and Secrets Manager

Storing SSH keys in Secrets Manager seems like an obvious choice when you need to securely manage private keys and make them available in serverless applications. I’ve found the hard way that it doesn’t always work as expected — and I’ve tried with keys used for SSH/SFTP, PGP encryption, and JWT sign/verify operations.

The issue is that some libraries, functions, and APIs are very picky about the format of the key. If they expect a PEM file, then it needs to have the line breaks just where they are supposed to be, and \n generally doesn’t work.

If we create an SSH key using:

ssh-keygen -P "" -m PEM -f test-key

Then the resulting PEM file will look like this:

 — — -BEGIN RSA PRIVATE KEY — — -
MIIG4wIBAAKCAYEAxrNZq+p2T02XvUkWI+PGANI1aJem3xVVwKSx/SJFUifEk3ah
8prCvlkMMgqTgYKBRG828HxT8tCeq0rpvszlXnan26/HpVUTfQMZM2YhEe6tsNuD
<<<snip....>>>
X5MIW2Jmf2ZQ9Pm+Pli3EnzsGJig23trsQMfrHtiWNBedCgVRxV8HpOkiSQoEIi+
edIpUenw0XSvfAw/o6P++RnQ+X+8YY+U0VfIq2S3lJtbSz+DsWRtjafAm4nf8VeG
8voBPqcmgo4Oxc/MsmSSF/IzRsceJOZSUV/67FzeyDrdEtCXBAJn
 — — -END RSA PRIVATE KEY — — -

We can then copy/paste it into Secrets Manager

Delete the JSON structure and paste the key into the Plaintext window

The problem is, when the Secrets Manager API returns the string, it looks like this:

aws secretsmanager get-secret-value --secret-id test-key --query SecretString --profile myprofile
>>
-----BEGIN RSA PRIVATE KEY-----\nMIIG4wIBAAKCAYEAxrNZq+p2T02XvUkWI+PGANI1aJem3xVVwKSx/SJFUifEk3ah\n8prCvlkMMgqTgYKBRG828HxT8tCeq0rpvszlXnan26 <<<<snip....>>>\nX5MIW2Jmf2ZQ9Pm+Pli3EnzsGJig23trsQMfrHtiWNBedCgVRxV8HpOkiSQoEIi+\nedIpUenw0XSvfAw/o6P++RnQ+X+8YY+U0VfIq2S3lJtbSz+DsWRtjafAm4nf8VeG\n8voBPqcmgo4Oxc/MsmSSF/IzRsceJOZSUV/67FzeyDrdEtCXBAJn\n-----END RSA PRIVATE KEY-----

Secrets Manager has helpfully inserted line breaks, but if you try to import that into the AWS Transfer API to update the server key it doesn’t work.

The solution is base64 encoding:

base64 test-key > test-key-b64

Copy/paste the resulting text into the Secret as we did before for the plain text key, and then Secrets Manager will return this:

aws secretsmanager get-secret-value — secret-id test-key — query SecretString — profile dev
>>
LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlHNHdJQkFBS0NBWUVBeHJOWnErcDJU\nMDJYdlVrV0krUEdBTkkxYUplbTN4VlZ3S1N4L1NKRlVpZkVrM2FoCjhwckN2bGtNTWdxVGdZS0JSnRzgyOEh4VDh0Q2VxMHJwdnN6bFhuYW4yNi9IcFZVVGZRTVpNMlloRWU2dHNOdUQKdVB6Si9INkx6naUlaM1JzYWRvODZJUmZwc0M1SExBN2RrU3gvUUZHbE1Gcno2dHgrUUo0TGtyUU1ZQlU0eDVybgp6nUTY1SU81T0tnZm1Oam1QdDhXb3RDUWxGOHRyT0hEUFIvU1RDL0FVelNJTDhLOFVCU1FhOXNYS2FW<<<snip...

Now Secrets Manager returns a single line of base64 text that we can work with.

If you are a command-line fan, you can test the whole process like this:

Create and upload an SSH Private key to Secrets Manager

Now you are ready to configure the server.

Lambda Custom Resource to Update The Host Key

The next issue is how to update the AWS Transfer Host Key in a CDK workflow. I have written about CDK AWS Custom Resources a few times — they are incredibly handy and easy to use if you need to make basic API calls to configure or customise a resource. In this case, however, we need to transform the base64 result from Secrets Manager before we send it to the API, so we need another solution.

Fortunately, it’s not a lot more difficult to create our own Lambda Custom Resource to get and transform the key, then update the AWS Transform server:

CDK code snippet and Lambda code for the custom resource

Lambda custom resources can get more complicated, but at its simplest, we just need to create/configure whatever it is we are doing and then return the PhysicalResourceId (or any unique Id if there isn’t a physical resource).

In this one all we are doing is getting the Base64 key string from Secrets Manager, converting back to ASCII, then sending it to the AWS Transfer updateServer API. Job done.

There are many other uses for the Secrets Manager Base64 encoding as I mentioned at the start. Just two I am also using include PGP keys and RSA keys for JWT signing/verifying. Hopefully, the tips here save you a little time for your own use cases.

There is more to do with AWS Transfer and I may add another chapter if there is any interest.

In particular, user management in Secrets Manager and refinement of the logical folder structure for users are where CDK can help with configuration. I will add another chapter at some point to cover them from a CDK point of view.

Please let me know below if you have any comments or suggestions.

About the Author

Mark is a 20+yr IT industry veteran, with technical expertise and entrepreneurial experience in a range of fields. For many years now his passion has been building on AWS, and he holds certifications including Solution Architect Professional and the Advanced Networking Specialty. He wants you to know that he is an infrastructure and architecture expert who writes code, not a professional software developer — just in case you spot something odd.

Mark has spent the last few years in the banking industry, where he is currently building and leading a team focused on using AWS serverless technologies to reduce costs and speed up the time to market for new services and products.

You can also find Mark on:

GitHub — https://github.com/markilott

LinkedIn — https://www.linkedin.com/in/markilott/

Twitter — https://twitter.com/mark_ilott

More content at plainenglish.io