Where does EC2 store User Data

EC2 User Data Explained

One of the great features of an AWS EC2 Linux (as wells as a Windows or Mac) instance is the concept of "user-data" which can be used to perform common automated configurations for your Linux box. I use them all the time to pre-install things like Docker, docker-compose, code-deploy agents, and CloudWatch configurations. You can also use to it automatically mount EBS volumes or S3 buckets using tools like s3sf, etc.

Where is it stored?

At some point, you may want to review the user data scripts installed on your EC2 Instance. But where does it actually reside? Technically it doesn't appear to be stored anywhere on your instance. Instead, it's stored in your metadata and pulled down and run during the instance set up. Although I haven't seen it specifically documented in the AWS docs; after that my best guess is it's simply discarded.

To view the user data linked to your EC2 instance you can use http://169.254.169.254/latest/user-data which is available from within your instance.

NOTE: If you're not familiar with the 169.254.169.254 address, it's an address used internally on all EC2 instances. If you clicked just now, it will be a dead link but within an EC2 instance, it will give you a lot of information. And you can use that address to get user data and meta-data.

The command line

From the command line within your instance you can use curl to see what is contained within each section. In this case we'll look at the user-data.

The Results

Running the command above will output your user data and show you something like the image below. In my case it's the script that installs docker and docker compose so that my EC2 instance is ready to run docker.

If you didn't add any user data during your launch configuration, then running the curl command above would simply return nothing.

The AWS Console

Note that you can't edit it unless the EC2 instance is in a stopped state

You should also be aware that the script will not be run on the next boot, which kind of begs the question of why can you edit it!?! I guess you could edit it, then use the curl http://169.254.169.254/latest/user-data to retrieve it and then manually run it.

Limitations

There are a few limitations with using user-data.

There is a limit to the size of 16 KB before it's base64-encoded (which is done for you in the aws console)
It's only run on the initial instance launch, so any modifications after the fact won't get executed.
If the script fails, there isn't much of a warning. Your instance will still start like normal (unless your script did something seriously descructive), so it will be up to you to check to make sure every runs smooth.

For complex setup and verifications you should consider solutions like ansible, puppet and chef - but for small quick setups keep user-data in mind!

Best Practices

Be sure to log the actions of your user-data scripts, so you can trouble shoot them later. The script below will send the actions performed to /var/log/user-data.log

I typically add a start time and end time to track how long the scripts take to run. This helps me gauge how quickly my instance will be in a healthy state.

#!/bin/bash -ex
# add user-data to a log file
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

# start time, which will also be used by the end.sh script
start_time="$(date -u +%s.%N)"

# add your scripts here 



# log the completed and
echo "Launch Script Completed"
end_time="$(date -u +%s.%N)"

elapsed="$(bc <<<"$end_time-$start_time")"
echo "Total of $elapsed seconds elapsed for script to run"

Examples

If you're interested in what user-data script might look like. Check out some of my other posts which are dedicated to specific examples of user-data scripts.