An Introduction to AWS Lambda Layers

When you're building AWS Lambda functions, especially in a language like Python, you'll quickly find yourself needing to include third-party libraries and dependencies. The standard way to do this is to package them all into a .zip file along with your function code. While this works, it can lead to large deployment packages, slow uploads, and duplicated code if you have multiple functions that share the same dependencies.

AWS Lambda Layers are the solution to this problem. A Layer is essentially a .zip archive that can contain libraries, a custom runtime, or other dependencies. By using Layers, you can separate your function's dependencies from your business logic.

Why Use Lambda Layers?

1. Smaller Deployment Packages: Your function's .zip file only needs to contain your code. All the heavy dependencies can be moved into a Layer. This makes your deployment packages much smaller and faster to upload.

2. Code Sharing and Reuse: If you have multiple Lambda functions that use the same set of libraries (e.g., requests, boto3, pandas), you can create a single Layer with these libraries and attach it to all of them. This promotes code reuse and simplifies dependency management.

3. Separation of Concerns: It creates a clean separation between your application code and its dependencies. This can make your project easier to manage, especially in a team environment.

How Lambda Layers Work

When your Lambda function is invoked, AWS downloads the attached Layers and extracts them to the /opt directory in the execution environment. Your function code can then access the libraries in that directory as if they were installed locally.

A function can use up to 5 Layers, and the total unzipped size of the function and all its Layers must not exceed the 250 MB unzipped deployment package size limit.

Creating a Python Dependency Layer

Let's walk through the most common use case: creating a Layer for Python packages.

1. Create a Directory Structure:

   Lambda expects a specific directory structure for the Python runtime. You need to place your packages inside a python directory.

   mkdir -p my-lambda-layer/python
   cd my-lambda-layer
   

2. Install Packages into the Directory:

   Use pip with the --target (-t) flag to install your desired packages into the python directory.

   pip install requests -t python/
   pip install pandas -t python/
   

   Your my-lambda-layer directory will now look something like this:

   my-lambda-layer/
   └── python/
       ├── requests/
       ├── pandas/
       └── ... (other dependency files)
   

3. Zip the Layer Content:

   Navigate back to the my-lambda-layer directory and zip the contents of the python directory. It's important to zip the contents, not the folder itself.

   cd python
   zip -r ../my-python-layer.zip .
   cd ..
   

4. Create the Lambda Layer in AWS:

   You can now create the Layer using the AWS CLI or the AWS Management Console.

   Using the AWS CLI:

   aws lambda publish-layer-version \
       --layer-name my-python-dependencies \
       --description "My common Python libraries" \
       --zip-file fileb://my-python-layer.zip \
       --compatible-runtimes python3.9 python3.10 python3.11
   

   This command will upload your .zip file and create a new Layer version. You'll get back a Layer Version ARN, which you'll need in the next step.

Attaching the Layer to a Lambda Function

Now that your Layer is created, you can attach it to any of your Lambda functions. You can do this in the function's configuration settings in the console, or through your Infrastructure as Code tool (like AWS CDK or Terraform).

Example with AWS CDK (TypeScript):

import * as lambda from 'aws-cdk-lib/aws-lambda';

// Reference an existing layer by its ARN
const myLayer = lambda.LayerVersion.fromLayerVersionArn(this, 'MyExistingLayer', 'arn:aws:lambda:us-east-1:123456789012:layer:my-python-dependencies:1');

new lambda.Function(this, 'MyFunction', {
    runtime: lambda.Runtime.PYTHON_3_11,
    handler: 'index.handler',
    code: lambda.Code.fromAsset('lambda'),
    layers: [myLayer] // Attach the layer here
});

Now, your Lambda function's code can simply import requests or import pandas, and it will work, even though those libraries are not in your function's .zip file.

Conclusion

AWS Lambda Layers are a powerful and essential tool for managing dependencies in a serverless environment. By separating your common libraries and dependencies from your function code, you can create smaller, more manageable deployment packages, promote code reuse, and streamline your entire development workflow. For any non-trivial serverless application, using Layers is a clear best practice.