AWS Lambda is the platform for deploying functions to the AWS cloud. You can use it to develop functions that respond to AWS events (eg S3 uploads, DynamoDB inserts), AWS API calls, or via HTTP endpoints using the API Gateway.
In this tutorial you’ll learn how to create a new Lambda function in Scala, accessible via an API Gateway HTTP endpoint but also testable locally with Docker. This tutorial has been tested with Scala 2.12.8, SBT 0.13.18, and SAM-CLI 0.15.0.
Update Since publishing this post I have found that setting the Lambda function memory size to 512 (mb) provides a significant reduction in response times for a “cold start” request. See the Measuring AWS Lambda Response Times post for details.
Requirements
This tutorial uses SBT for building Scala projects and Amazon’s SAM CLI app for testing, packaging, and deploying AWS Lambda functions. You’ll need all of the following to complete the tutorial:
- SBT • The Scala Build Tool
- SAM CLI • The AWS Serverless Application Model app (a wrapper around AWS CloudFormation and the CLI)
- AWS CLI • The popular CLI for accessing the AWS API
- Docker • Required by SAM for local testing of Lambda functions
- curl • The performant web client used to test the running Lambda functions
- An AWS account and valid credentials
The Project
Getting The Project
Download and extract the aws-lambda-hello-scala repository.
It’s a small repo, so if you want to get more familiar with the layout of Scala projects you can opt to browse the repo and type them in yourself.
Along with the standard Git project files (.gitignore and README.md), there are
three build files (build.sbt, project/assembly.sbt, and project/build.properties),
a SAM configuration file (template.yaml), a Java entrypoint (ApiGatewayProxyHandler.java),
and the single Scala function (ApiHandler.scala). The important files are described in more detail in the following section.
Got the project? Run sbt assembly to build & package & verify you’re ready for the rest of the tutorial. The project specifies the version of Scala and SBT used so the assembly step should “just work”.
About The SBT Build
The build.sbt file includes the project name, Scala version, and the two Java AWS-Lambda libraries used to handle the incoming API Gateway events. These lines show the two Java library dependencies:
6 | libraryDependencies ++= Seq( |
The sbt-assembly plugin (added in project/assembly.sbt) is used to build the full application and all dependent libraries into a single “fat” JAR file. This line defines the name of the fat JAR file generated by sbt-assembly:
4 | assemblyJarName in assembly := "hello-scala.jar" |
Finally the build.properties file defines the SBT version used, 0.13.18.
About The Java Entrypoint
The AWS Lambda platform defines a number of available runtimes including Node.js, Python, Go and Java, but not one for Scala directly. Fortunately SBT does compile Java files, so we can use a Java class as our entrypoint and then immediately invoke the Scala function.
Here is the single Java function handleRequest(). It takes the API Gateway request and context, calls our Scala function, and returns a standard API Gateway response.
13 | /** |
Our Scala function, ApiHandler.handle(), will take the Java APIGatewayProxyRequestEvent and Context instances and return a Scala ApiHandler.Response which this function handily copies into the APIGatewayProxyResponseEvent return value.
As an improvement, one could make this entrypoint even friendlier to our Scala function and convert the Java arguments and their Java collections to Scala instances and collections. I left that part out (there’s a lot of data to convert in those instances!), so the Scala code will have some odd-looking Java collection calls.
About The Scala Handler
And now for the good part! Here is the single Scala function handle() which takes the aforementioned Java instances and returns an instance of the case class Response.
12 | object ApiHandler { |
This function handles the business logic for our sample Lambda function, namely:
- Printing out some useful information about the request and the context, including the amount of time remaining until the Lambda execution times out.
- Printing out an environment variable that we’ll set in the
template.yamlSAM configuration file, just to demonstrate how they can be used. - Retrieving the
namepath parameter passed to our function - Returning a response including the body (“Hello,
“) and content type.
You’ll be able to view the output of these println statements when running the function, both locally and in AWS. The Context instance does provide its own logging features, but since Scala’s println works I’d rather keep things simple and log output the Scala way.
Note the Response.javaHeaders() helper function in the case class. This allows us to use concise Scala syntax to build the map of HTTP headers while providing a java.util.Map collection for use in the Java API response instance.
About The SAM Configuration
The SAM CLI can use a single yaml file to define both the AWS Lambda function and the API Gateway endpoint configuration. This is a wonderfully simple way to define a Lambda function, where the API Gateway is implicitly configured from the Lambda function(s) definition. A more explicit approach is to also define the API Gateway endpoints in this file with an OpenAPI Specification (a.k.a Swagger), but for a simple function this will seem like a duplication of effort.
Note: a drawback to using the streamlined implicit configuration is that (as of SAM CLI 0.15.0) the API Gateway’s StageName variable will be set to its default value of Prod and thus be set in the first section of the external endpoint path . If you want a single function definition that can be deployed to multiple stages and not have Prod in the path you’ll need to add the API Gateway section with an OpenAPI specification to the file.
Here is the full SAM configuration with the streamlined implicit API Gateway configuration:
1 | AWSTemplateFormatVersion: '2010-09-09' |
Some of the important features to point out:
- As is standard for Java Lambda functions the template specifies the java8 runtime, the full canonical (package and name) of the Java entrypoint, and the path to the “fat” JAR file.
- The max return time of the Lambda function is increased from the default 3 seconds to 15 seconds to give the Lambda engine enough time to cold-start this Java application (typically about 10 seconds).
- SAM will hard-code the API Gateway’s
StageNametoProdwhich figures into the deployed endpoint path. To work around this inability to change theStageName, I’ve introduced a separate environment variable simply namedenvas an example of an alternate strategy for communicating development stages to your deployed functions. - The
ApiURLoutput variable is using CloudFormation’s substitution function to build the eventual url of the Lambda function. We’ll use the AWS CLI to read this generated url after deploying the function to AWS.
Execution
At this point you should have downloaded or otherwise created the “aws-lambda-hello-scala” project and are ready to try it out. We’ll be running and testing out the Lambda function first locally in Docker and then remotely deployed to AWS.
Running The Function Locally
Open your terminal to the project directory and run the following to build and run the function locally:
$ sbt assembly && sam local start-apiThe sbt assembly command to compile the function and package it in a “fat” JAR file could be run by itself, but since sam local depends on this step it’s easier and clearer to combine them and then fail immediately if there is a compilation problem.
Open a separate terminal in the same directory and test out the function with curl:
$ curl http://127.0.0.1:3000/hello/developer
Hello, developerThe Hello, $name response from the function appears, with the name taken from the path parameter following the hello/ segment.
Switch back to the first terminal (the one running sam local start-api) and you’ll see the output from the Scala function’s println statements.
handling GET /hello/developer, remaining time is 14202 ms
environment = stagingIn addition to the method and path, the request instance contains methods for accessing the headers, query string parameters, the POST body…. in short, everything you’ll need for an HTTP endpoint.
Now that the function is verified as working locally, it’s time to deploy it to AWS Lambda!
Deploying The Function To AWS
The deployment process requires interaction with AWS APIs and building infrastructure, and so is complicated enough to warrant being explained in multiple steps.
Create An S3 Bucket
To package and deploy your function to AWS you’ll need an S3 Bucket to hold the packaged (zipped) function code. Head over to the AWS S3 Console and create a new S3 bucket or select an existing S3 bucket to hold the code during deployment. Save the bucket name (just the name, not the full S3 URI) for the upcoming deploy step.
Upload Your Function
Run the following command to package and upload the function to your S3 bucket. You should already have the SAM CLI app installed (see the requirements section above).
$ sam package --s3-bucket <your-s3-bucket-name> --output-template-file packaged.yaml
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /<path>/packaged.yaml --stack-name <YOUR STACK NAME>The “Uploading” progress meter should finish up at 100%, followed by this instruction you can disregard about deploying the package with cloudformation (a reminder that the SAM CLI is essentially a utility wrapper around AWS CloudFormation and the AWS CLI). The result of this deployment is a packaged version of your function sitting in S3 and a generated packaged.yaml CloudFormation file ready for deployment.
Pick An AWS Region
While S3 buckets are global, AWS Lambda functions and API Gateway endpoints are regional. Before deploying your function you will need to choose your AWS Region. When you have chosen a region (perhaps one close to you, or with a preferred pricing model) and its region code (eg “us-west-2”), run the following command to set your region for AWS operations:
$ export AWS_DEFAULT_REGION=<region-code>Deploy Your Function
Run the following command to deploy your function from S3 to AWS Lambda in your selected region:
$ sam deploy --template-file packaged.yaml --stack-name aws-lambda-hello-scala --capabilities CAPABILITY_IAM
Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - aws-lambda-hello-scalaAfter about a minute you should see the confirmation message that the CloudFormation stack “aws-lambda-hello-scala” has been created (and executed!). You now have an API Gateway endpoint and Lambda function deployed.
Retrieve the Endpoint URL
The next step is to find the url of that new endpoint. Fortunately an output command was included in the template.yaml (and thus packaged.yaml) to print out the new url (see the About The SAM Configuration section above).
Run the following command to describe the CloudFormation stack including the rendered url:
$ aws cloudformation describe-stacks --stack-name aws-lambda-hello-scala
"Stacks": [
{
"StackId": "arn:aws:cloudformation...:stack/aws-lambda-hello-scala/...",
"StackName": "aws-lambda-hello-scala",
"ChangeSetId": "arn:aws:cloudformation:...:changeSet/awscli-cloudformation-package-deploy..",
"Description": "A SAM CLI template for a single-endpoint Hello World Lambda function in Scala",
"StackStatus": "CREATE_COMPLETE",
"Outputs": [
{
"OutputKey": "ApiURL",
"OutputValue": "https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/{name}",
"Description": "API endpoint URL for Prod environment"
}
],..
}
} Test Your Function
It’s time to try out your function running in AWS Lambda! You can use the same
curl command and “name” argument as you used last time, plus the time command to measure the function’s performance.
Run the following command, using the actual hostname in the OutputValue field returned from your previous command:
$ time curl https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer
curl 0.02s user 0.01s system 1% cpu 10.57 totalYou can see the response is correct, but the performance (at 10+ seconds) isn’t that great. This is an example of the famous “AWS Lambda Cold Start” issue where the first execution after a deploy, or the first execution after the function has “slept” after a short period of unused, includes time to “wake up” the function and start the Java runtime.
Run the command again a few times and you’ll see far more reasonable response times:
$ time curl https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer
curl 0.02s user 0.01s system 1% cpu 0.23 total
$ time curl https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer
curl 0.02s user 0.01s system 1% cpu 0.15 totalWhen the AWS Lambda Java runtime is fired up and hasn’t been put to sleep after a certain amount of inactivity the response time is quite reasonable - here returning in 230ms and 150ms. If you do not expect your function to have regular periods of inactivity, or find occasional responses of 10 seconds after inactivity acceptable, then you should be all set. Otherwise, there are recommendations available for dealing with the “cold starts” that include sending “wake-up” traffic at regular intervals to avoid the Java runtime “sleeping” and other solutions.
As you recall from testing the function locally, the Scala handler code included a few println() commands to log the function’s request and context information. Fortunately in AWS Lambda these statements have been logged to AWS Cloudwatch.
You can view your function’s logs by running the following command:
$ sam logs -n HelloScalaFunction --stack-name aws-lambda-hello-scala
Found credentials in shared credentials file: ~/.aws/credentials
START RequestId: 3426d1e6-3843-434a-8564-a5a6509aad93 Version: $LATEST
handling GET /hello/developer, remaining time is 14999 ms
environment = staging
END RequestId: 3426d1e6-3843-434a-8564-a5a6509aad93
REPORT RequestId: 3426d1e6-3843-434a-8564-a5a6509aad93 Duration: 1.81 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 115 MB This is very much like the output from running the AWS Lambda function locally (in docker), with the extra information about the API Gateway request.
Congratulations! You have now verified your AWS Lambda function is deployed and working correctly!
Wrapping Up
Decommissioning Your Lambda Function
I hope you’ll continue to experiment with writing AWS Lambda functions in Scala. This particular function you’ve deployed is only really useful for learning and testing. You could choose to let it stay deployed and keep an eye on the relatively-inexpensive costs with AWS Cost Explorer. I prefer to shut down AWS resources when I’m done with them, as I tend to forget about them when I don’t use them.
When you are ready to shut the AWS resources deployed in this article (besides the S3 bucket you may have created), you can do so by deleting the CloudFormation stack defined by template.yaml. This will remove the API Gateway endpoint, undeploy your AWS Lambda function, and (obviously) remove the CloudFormation stack.
Run this command to delete the CloudFormation stack and its resources:
$ aws cloudformation delete-stack --stack-name aws-lambda-hello-scalaAfter a few seconds you can check with CloudFormation to verify the stack was deleted:
$ aws cloudformation describe-stacks --stack-name aws-lambda-hello-scala
An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id aws-lambda-hello-scala does not existMore directly, you can try to hit your endpoint with the curl command again and verify it fails:
$ curl https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
# while the hostname is still active but the endpoint / function are decommissioned
{"message": "Internal server error"}
# after the hostname has been decommissioned
curl: (6) Could not resolve host: f2c0a1aa55.execute-api.us-west-2.amazonaws.comNext Steps
The 10-second “cold start” problem with the Java runtime in AWS Lambda functions may be a blocker for many developers. Although it is possible to keep the Java runtime active and awake, this does incur a (minor) additional cost for the traffic. A potential solution worth exploring is using the GraalVM to compile a Scala function to a native executable and then invoke it with the custom AWS Lambda runtime. The startup time for a GraalVM native Scala app should be in the milliseconds instead of ~10 seconds and remove the “cold start” problem altogether. One challenge is that the custom runtime requires extra work to use, as the function needs to do the work of fetching requests instead of being called from an entrypoint.
Another improvement you could explore is adding a AWS::Serverless::Api section to the template.yaml file in which you can override the StageName to something other than “Prod”. This could be useful for maintaining separate deployment environments where the environment name is included in the url. For example you could define stages that keep your deployed versions separate (eg /dev/hello/{name} vs /staging/hello/{name} vs /prod/hello/{name}).
Conclusion
The AWS SAM CLI provides a straightforward way to build, test, and deploy AWS Lambda functions in Scala (albeit with a Java entrypoint). You can reuse the aws-lambda-hello-scala example project as a starting point for your own projects and explore the limits of serverless functions!