TL;DR: Cross-compiling Scala functions to Linux executables in Docker using GraalVM and executing them in AWS Lambda with the custom (aka “provided”) runtime is found to provide an average performance of 150ms (increased to 250ms for “cold starts”), a clear improvement over executing Scala functions with the AWS Lambda’s Java runtime.
In the last month I have covered building AWS Lambda functions with Scala (using the Java runtime) and measuring AWS Lambda function cold & warm response times. The former provided an introduction to writing, testing and deploying Scala functions with the AWS SAM cli. The latter showed how applying ever-increasing values of the MemorySize setting for AWS Lambda functions on the Java runtime significantly reduces “cold start” response times.
In this post I’ll show how your AWS Lambda functions in Scala can achieve even greater performance by compiling them to native Linux executables with GraalVM. AWS Lambda supports running Linux apps with its custom runtime, in which the app is supposed to loop, pull requests from one endpoint, and write the responses out to another endpoint.
The code has been tested on OS X. Linux users will be able to bypass the Docker step and build the executables natively with the GraalVM package. Windows users - the build.sh script should run in cygwin or Windows terminal, or could be replaced by a Powershell script - please let me know how this works.
Requirements
This post uses SBT for building Scala projects, Docker for further compiling the Scala code to a linux executable, and Amazon’s SAM CLI app for testing/packaging/deploying AWS Lambda functions. You’ll need all of the following to run the project:
- SBT
- The Scala Build Tool
- SAM CLI
- The AWS Serverless Application Model app (a wrapper around AWS CloudFormation and the CLI)
- AWS CLI
- The popular CLI for accessing the AWS API
- Docker
- Required by SAM for local testing of Lambda functions
- curl
- The performant web client used to test the running Lambda functions
- An AWS account and valid credentials
- Allows the SAM tool to deploy your function to API Gateway / AWS Lambda
The Project
Clone or download the graalvm-scala-lambda project. Along with the standard Git project files (.gitignore and README.md), there are
four build files (build.sh, build.sbt, project/assembly.sbt, and project/build.properties),
a SAM configuration file (template.yaml), a dockerfile for building Linux executables from any system (linuxbuild.dockerfile), and two Scala files (bootstrap.scala and AWSLambdaTypes.scala) containing the custom runtime handler and its case classes.
The project is very similar to the aws-lambda-hello-scala project as covered in the recent building AWS Lambda functions with Scala post. To avoid duplicating much of that previous post I’ll stick to covering the new and shiny parts in this post, and will assume you’ve reviewed the original project and mostly want to read about the new, shiny parts.
About The SBT Build
The build.sbt file includes Li Haoyi’s wonderfully-simple Requests library for interacting with the custom runtime’s endpoints over HTTP and Andriy Plokhotnyuk’s fast-as-hell Jsoniter library for marshalling & unmarshalling case classes to/from JSON.
7 | libraryDependencies ++= Seq( |
The sbt-assembly plugin is used to build the single “fat” JAR file, an intermediate component in the process to build out a Linux executable. Here’s the sbt-assembly settings for the name of the generated JAR file and its main class (also the main class for the compiled executable):
4 | assemblyJarName in assembly := "graalvm-scala-lambda.jar" |
About The Scala App
The AWS Lambda custom runtime will boot the native executable and expect it to call internal endpoints iteratively, reading requests and writing responses. The executable can support multiple “functions”, ie API Gateway endpoints, so it works more like a command-line application than a serverless function.
Here’s the bootstrap Scala application:
1 | object bootstrap { |
The fact that this is a command-line app that handles requests by polling an endpoint should explain why this looks
nothing like the Scala or Java entrypoints in the building AWS Lambda functions with Scala post.
It has a main() command-line entry point which reads the runtime’s host name from an environment variable, a handleEvents()
method which iteratively retrieves and parses the next request, a handleRequest() method to process a single request,
and a collection of helper functions.
The lower case object name bootstrap may look odd, as Scala type names are typically capitalized. The lower case name is used, however, to ensure the object, JAR file main class, and linux executable file name will all have the correct case for case-sensitive filesystems.
The second of the two Scala files contains the types for unmarshalling the request payload and marshalling the response payload, plus JSON helper functions to handle conversion quickly and safely.
1 | import scala.util.Try |
The RequestEvent type and sub-types were designed for compatibility with the JSON payloads retrieved from the next-event url, tested with a variety of endpoint configurations. It isn’t fully clear from the AWS documentation which fields are required, so your mileage (ie, success with parsing the JSON payload) may vary. This should explain all the optional fields. The JSONiter library is excellent for performance, but throws exceptions on missing fields, which is why it is wrapped in a Try in the fromJsonSafe() method.
The LambdaResponse type was designed based on the official Output Format of a Lambda Function for Proxy Integration, also tested with a variety of responses for compatibility. I’ve set isBase64Encoded to false as a default as the tutorial doesn’t return binary downloads. Another possible candidate for a default value is statusCode = "200" (I don’t know why API Gateway expects a JSON String!.
Another way to return errors is with the custom runtime’s Invocation Error URL. I couldn’t get this working locally with the SAM tool, so left it off this tutorial; regular HTTP error codes seem to work fine locally with SAM as well as deployed to AWS Lambda.
That’s all the Scala code - now its time to build a Linux app!
About The Linux Compilation
The AWS Lambda custom runtime (unlike the Java/Python/Node.js/Ruby runtimes) requires a Linux executable or shell script to run. I’m using the the GraalVM native-image app to build a native executable from the (SBT-built) JAR file, and a Docker container to allow building Linux executables from a non-Linux environment (OS X in my case).
Here’s the dockerfile for building the Linux executable, based on the official GraalVM Docker base image :
1 | FROM oracle/graalvm-ce:1.0.0-rc16 |
The GraalVM Docker base image provides the Linux OS (oraclelinux, specifically) and the GraalVM tools. The input directory (/tmp/target) and output directory (/tmp/dist) will be mapped to local directories when the docker image is executed, enabling us to retrieve the generated executable. The --enable-url-protocols=http argument is required for http support by GraalVM which otherwise leaves it out to optimize the executable size.
A build script ties together SBT and the GraalVM build in Docker:
1 |
|
Mounting the target/scala-2.12 directory keeps the JAR file out of the docker image, which means you can build the docker once locally and simply run lines 7 and 9 while compiling new iterations of your code. Mounting the dist directory allows us to retrieve the executable file generated when the docker image is run, the entire point of this exercise.
Go ahead and run the build script to generate the Linux executable. Note that the GraalVM docker image is a hefty download so the first execution may take more than the standard 2 minutes to run.
$ ./build.sh
[info] Loading project definition from /graalvm-scala-lambda/project
[info] Set current project to graalvm-scala-lambda (in build file:/graalvm-scala-lambda/)
...
[success] Total time: 14 s, completed May 22, 2019 9:05:06 AM
Sending build context to Docker daemon 38.79MB
..
Successfully tagged linuxbuild:latest
Build on Server(pid: 10, port: 42035)*
[bootstrap:10] classlist: 9,268.85 ms
...
[bootstrap:10] [total]: 60,824.79 msThe output app, dist/bootstrap, is now ready to be tested locally with SAM.
About The SAM Configuration
Following the convention in the previous post I’m using SAM’s implicit endpoint configuration in template.yaml to define the AWS Lambda function and API Gateway endpoints togther. This has the same drawback as before, where the API Gateway StageName is set to Prod by SAM for implicit configurations, leading to a curious Prod mention in the endpoint path. If you prefer to keep production and non-production functions in separate AWS accounts, as I do, this is more an oddity than a problem.
Here is the full SAM configuration, which uses the custom runtime (“provided”) and points to the bootstrap executable in the dist local directory:
1 | AWSTemplateFormatVersion: '2010-09-09' |
The ApiURL output variable will generate the eventual url of the Lambda function when the CloudFormation stack is executed, allowing us to retrieve it with the AWS CLI.
Execution
Running The Function Locally
Now that you’ve run build.sh and generated a Linux executable, it’s time to try it out with the SAM CLI. Run the following command to start it up:
$ sam local start-apiThen, in a separate terminal, test it out with the curl command:
$ curl http://127.0.0.1:3000/hello/developer
Hello, developerYou should see the Hello, $name response from line 40 of src/main/scala/bootstrap.scala.
Now that the function is verified as working locally, it’s time to deploy it to AWS Lambda!
Deploying The Function To AWS
Create An S3 Bucket
To package and deploy your function to AWS you’ll need an S3 Bucket to hold the packaged (zipped) function code. Head over to the AWS S3 Console and create a new S3 bucket or select an existing S3 bucket to hold the code during deployment. Save the bucket name (just the name, not the full S3 URI) for the upcoming deploy step.
Upload Your Function
Run the following command to package and upload the function to your S3 bucket:
$ sam package --s3-bucket <your-s3-bucket-name> --output-template-file packaged.yamlPick An AWS Region
While S3 buckets are global, AWS Lambda functions and API Gateway endpoints are regional. Before deploying your function you will need to choose your AWS Region. When you have chosen a region (perhaps one close to you, or with a preferred pricing model) and its region code (eg “us-west-2”), run the following command to set your region for AWS operations:
$ export AWS_DEFAULT_REGION=<region-code>Deploy Your Function
Run the following command to deploy your function from S3 to AWS Lambda in your selected region:
$ sam deploy --template-file packaged.yaml --stack-name hello-scala-graal --capabilities CAPABILITY_IAM
Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - hello-scala-graalAfter about a minute you should see the confirmation message that the CloudFormation stack “aws-lambda-hello-scala” has been created (and executed!). You now have an API Gateway endpoint and Lambda function deployed.
Retrieve the Endpoint URL
Run the following command to retrieve the rendered ApiURL output variable containing the Lambda’s endpoint (unnecessary details hidden):
$ aws cloudformation describe-stacks --stack-name hello-scala-graal
{
"Stacks": [
...
"StackName": "hello-scala-graal",
...
"StackStatus": "CREATE_COMPLETE",
...
"Outputs": [
{
"OutputKey": "ApiURL",
"OutputValue": "https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/{name}",
"Description": "API endpoint URL for Prod environment"
}
],
...
]
}Test Your Deployed Function
Run the following command (using the actual hostname from the OutputValue above) to try out your deployed function and also measure its performance:
$ time curl https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer
curl 0.02s user 0.01s system 15% cpu 0.69s totalNow that the custom runtime has been verified as deployed and is warmed up (ie there is at least one instance of the function that is running in a continuous loop), run the command a few more times to verify the typical response time. You should see performance similar to these results:
$ time curl https://vwtd2iu524.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer!
curl 0.02s user 0.01s system 19% cpu 0.147 total
$ time curl https://vwtd2iu524.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer!
curl 0.02s user 0.01s system 15% cpu 0.186 total
$ time curl https://vwtd2iu524.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer!
curl 0.02s user 0.01s system 19% cpu 0.153 totalWarmed up, the native function returns in an averaged 170 milliseconds.
Decommissioning Your Lambda Function
When you are finished with your function, you can decommission the API Gateway endpoint and AWS Lambda with the following CloudFormation CLI command:
$ aws cloudformation delete-stack --stack-name hello-scala-graalAfter a few seconds you can verify the resources have been deleted by calling describe-stacks again:
$ aws cloudformation describe-stacks --stack-name hello-scala-graal
An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id hello-scala-graal does not existOptimizing Native Function Performance By Memory Allocation
In a recent post I write about measuring AWS Lambda function cold & warm response times for functions deployed with the Java runtime, which clearly benefits from increased memory sizes. I performed similar tests for the native function with the custom runtime covered in this post, with one improvement to “cold start” measurements. To better measure “cold start” times I added a sleep time of 6 minutes to the script and then accessed the function. I’m assuming this is a better test of the period after which AWS Lambda puts the runtime to “sleep” than the previous methodology of testing the function immediately after it is deployed, as “cold starts” are expected to occur after periods of inactivity.
Here’s the result of the response time per MemorySize measuring for this native function:
As you can see, other than the additional jump for the 256Mb setting, increase the allocated memory from 128Mb (minimum) to 3008Mb (maximum) had little effect on the response times. The “cold start” (ie, 6 minutes between accesses) response times averaged about 250ms while accesses separated by mere seconds averaged about 150ms.
Next Steps
Some additional areas to consider for performance gains may include:
- Optimizing the code for reading and unmarshalling requests
- Optimizing the code for marshalling and writing responses
- Trying out different cross-compiling solutions such as Scala-Native or the upcoming production release of GraalVM
- Using a natively-compiled language (eg C, Rust) for the network features and JSON handling
Conclusion
The GraalVM compiler together with the AWS Lambda custom runtime provide a high-performance solution for running Scala code in AWS Lambda than using the Java runtime. At 250ms for “cold starts” and 150ms for all other responses (albeit with an overly-simplified “Hello, World” function), this combination should provide an suitably performant platform for Scala functions in production.
