Black Rock Blog
AWS Lambda, Scala, and GraalVM | BKS2

AWS Lambda, Scala, and GraalVM

Posted on | In , , | Comments:

TL;DR: Cross-compiling Scala functions to Linux executables in Docker using GraalVM and executing them in AWS Lambda with the custom (aka “provided”) runtime is found to provide an average performance of 150ms (increased to 250ms for “cold starts”), a clear improvement over executing Scala functions with the AWS Lambda’s Java runtime.

In the last month I have covered building AWS Lambda functions with Scala (using the Java runtime) and measuring AWS Lambda function cold & warm response times. The former provided an introduction to writing, testing and deploying Scala functions with the AWS SAM cli. The latter showed how applying ever-increasing values of the MemorySize setting for AWS Lambda functions on the Java runtime significantly reduces “cold start” response times.

In this post I’ll show how your AWS Lambda functions in Scala can achieve even greater performance by compiling them to native Linux executables with GraalVM. AWS Lambda supports running Linux apps with its custom runtime, in which the app is supposed to loop, pull requests from one endpoint, and write the responses out to another endpoint.

The code has been tested on OS X. Linux users will be able to bypass the Docker step and build the executables natively with the GraalVM package. Windows users - the build.sh script should run in cygwin or Windows terminal, or could be replaced by a Powershell script - please let me know how this works.

Requirements

This post uses SBT for building Scala projects, Docker for further compiling the Scala code to a linux executable, and Amazon’s SAM CLI app for testing/packaging/deploying AWS Lambda functions. You’ll need all of the following to run the project:

  1. SBT
    • The Scala Build Tool
  2. SAM CLI
    • The AWS Serverless Application Model app (a wrapper around AWS CloudFormation and the CLI)
  3. AWS CLI
    • The popular CLI for accessing the AWS API
  4. Docker
    • Required by SAM for local testing of Lambda functions
  5. curl
    • The performant web client used to test the running Lambda functions
  6. An AWS account and valid credentials
    • Allows the SAM tool to deploy your function to API Gateway / AWS Lambda

The Project

Clone or download the graalvm-scala-lambda project. Along with the standard Git project files (.gitignore and README.md), there are four build files (build.sh, build.sbt, project/assembly.sbt, and project/build.properties), a SAM configuration file (template.yaml), a dockerfile for building Linux executables from any system (linuxbuild.dockerfile), and two Scala files (bootstrap.scala and AWSLambdaTypes.scala) containing the custom runtime handler and its case classes.

The project is very similar to the aws-lambda-hello-scala project as covered in the recent building AWS Lambda functions with Scala post. To avoid duplicating much of that previous post I’ll stick to covering the new and shiny parts in this post, and will assume you’ve reviewed the original project and mostly want to read about the new, shiny parts.

About The SBT Build

The build.sbt file includes Li Haoyi’s wonderfully-simple Requests library for interacting with the custom runtime’s endpoints over HTTP and Andriy Plokhotnyuk’s fast-as-hell Jsoniter library for marshalling & unmarshalling case classes to/from JSON.

build.sbt
7
8
9
10
11
libraryDependencies ++= Seq(
  "com.lihaoyi" %% "requests" % "0.1.8",
  "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core" % "0.47.0" % Compile, 
  "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "0.47.0" % Provided // required only in compile-time
)

The sbt-assembly plugin is used to build the single “fat” JAR file, an intermediate component in the process to build out a Linux executable. Here’s the sbt-assembly settings for the name of the generated JAR file and its main class (also the main class for the compiled executable):

build.sbt
4
5
assemblyJarName in assembly := "graalvm-scala-lambda.jar"
mainClass in assembly := Some("bootstrap")

About The Scala App

The AWS Lambda custom runtime will boot the native executable and expect it to call internal endpoints iteratively, reading requests and writing responses. The executable can support multiple “functions”, ie API Gateway endpoints, so it works more like a command-line application than a serverless function.

Here’s the bootstrap Scala application:

src/main/scala/bootstrap.scala
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
object bootstrap {

  def main(args: Array[String]): Unit = {
    val runtimeApiHost = sys.env("AWS_LAMBDA_RUNTIME_API")
    handleEvents(runtimeApiHost)
  }

  // Receive and handle events infinitely
  def handleEvents(runtimeApiHost: String): Unit = {
    val nextEventUrl = getNextEventUrl(runtimeApiHost)
    while (true) {
      val r = requests.get(nextEventUrl)
      val statusCode = for {
        requestEvent <- RequestEvent.fromJsonSafe(r.text)
        deadlineMs <- getRequiredHeader(r, "lambda-runtime-deadline-ms").map(_.toLong)
        requestId <- getRequiredHeader(r, "lambda-runtime-aws-request-id")
        statusCode = handleRequest(runtimeApiHost, requestEvent, requestId, deadlineMs)
      } yield statusCode
      
      if (statusCode.isEmpty) {
        Console.err.println(s"Could not handle event: $r")
      }
    }
  }
  
  
  // Retrieve the specified header from the response. If not available, report the header as missing  
  def getRequiredHeader(nextEventResponse: requests.Response, header: String): Option[String] = {
    nextEventResponse.headers.get(header).map(_.head).orElse {
      Console.err.println(s"Next event request did not include this required header: $header")
      Console.err.println(s"headers: ${nextEventResponse.headers}")
      None
    }
  }
  
  // Handle a valid request to this function
  def handleRequest(host: String, requestEvent: RequestEvent, requestId: String, deadlineMs: Long): Int = {
    requestEvent.pathParameters.flatMap(_.get("name")) match {
      case Some(name) =>
        val response = LambdaResponse("200", Map("Content-Type" -> "text/plain"), s"Hello, $name!\n")
        requests.post(getResponseUrl(host, requestId), data = response.toJson).statusCode
      case None =>
        val response = LambdaResponse("400", Map("Content-Type" -> "text/plain"), "'name' param not found\n")
        requests.post(getResponseUrl(host, requestId), data = response.toJson).statusCode
    }
  }

  // The url used to retrieve the next function request
  def getNextEventUrl(host: String) =
    s"http://$host/2018-06-01/runtime/invocation/next"

  // The url used to write a response back to the caller
  def getResponseUrl(host: String, requestId: String) =
    s"http://$host/2018-06-01/runtime/invocation/$requestId/response"

}

The fact that this is a command-line app that handles requests by polling an endpoint should explain why this looks nothing like the Scala or Java entrypoints in the building AWS Lambda functions with Scala post. It has a main() command-line entry point which reads the runtime’s host name from an environment variable, a handleEvents() method which iteratively retrieves and parses the next request, a handleRequest() method to process a single request, and a collection of helper functions.

The lower case object name bootstrap may look odd, as Scala type names are typically capitalized. The lower case name is used, however, to ensure the object, JAR file main class, and linux executable file name will all have the correct case for case-sensitive filesystems.

The second of the two Scala files contains the types for unmarshalling the request payload and marshalling the response payload, plus JSON helper functions to handle conversion quickly and safely.

src/main/scala/AWSLambdaTypes.scala
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
import scala.util.Try

import com.github.plokhotnyuk.jsoniter_scala.core._
import com.github.plokhotnyuk.jsoniter_scala.macros._


case class RequestIdentity(
  apiKey: Option[String],
  userArn: Option[String],
  cognitoAuthenticationType: Option[String],
  caller: Option[String],
  userAgent: Option[String],
  user: Option[String],
  cognitoIdentityPoolId: Option[String],
  cognitoAuthenticationProvider: Option[String],
  sourceIp: Option[String],
  accountId: Option[String],
)

case class RequestContext(
  resourceId: String,
  apiId: String,
  resourcePath: String,
  httpMethod: String,
  accountId: String,
  stage: String,
  identity: RequestIdentity,
  extendedRequestId: Option[String],
  path: String
)

// The request returned from the next-event url
case class RequestEvent(
  httpMethod: String,
  body: Option[String],
  resource: String,
  requestContext: RequestContext,
  queryStringParameters: Option[Map[String, String]],
  headers: Option[Map[String, String]],
  pathParameters: Option[Map[String, String]],
  stageVariables: Option[Map[String, String]],
  path: String,
  isBase64Encoded: Boolean
)

object RequestEvent {
  private implicit val codec: JsonValueCodec[RequestEvent] = JsonCodecMaker.make[RequestEvent](CodecMakerConfig())

  def fromJsonSafe(s: String): Option[RequestEvent] = Try(readFromString[RequestEvent](s)) match {
    case util.Success(re) => Some(re)
    case util.Failure(ex) => Console.err.println(s"Failed to parse body into RequestEvent: $ex \nbody: $s"); None
  }
}

// The response written to the response url by the function
case class LambdaResponse(
  statusCode: String,
  headers: Map[String, String],
  body: String,
  isBase64Encoded: Boolean = false
) {
  private implicit val codec: JsonValueCodec[LambdaResponse] = JsonCodecMaker.make[LambdaResponse](CodecMakerConfig())

  def toJson: String = writeToString(this)
}

The RequestEvent type and sub-types were designed for compatibility with the JSON payloads retrieved from the next-event url, tested with a variety of endpoint configurations. It isn’t fully clear from the AWS documentation which fields are required, so your mileage (ie, success with parsing the JSON payload) may vary. This should explain all the optional fields. The JSONiter library is excellent for performance, but throws exceptions on missing fields, which is why it is wrapped in a Try in the fromJsonSafe() method.

The LambdaResponse type was designed based on the official Output Format of a Lambda Function for Proxy Integration, also tested with a variety of responses for compatibility. I’ve set isBase64Encoded to false as a default as the tutorial doesn’t return binary downloads. Another possible candidate for a default value is statusCode = "200" (I don’t know why API Gateway expects a JSON String!.

Another way to return errors is with the custom runtime’s Invocation Error URL. I couldn’t get this working locally with the SAM tool, so left it off this tutorial; regular HTTP error codes seem to work fine locally with SAM as well as deployed to AWS Lambda.

That’s all the Scala code - now its time to build a Linux app!

About The Linux Compilation

The AWS Lambda custom runtime (unlike the Java/Python/Node.js/Ruby runtimes) requires a Linux executable or shell script to run. I’m using the the GraalVM native-image app to build a native executable from the (SBT-built) JAR file, and a Docker container to allow building Linux executables from a non-Linux environment (OS X in my case).

Here’s the dockerfile for building the Linux executable, based on the official GraalVM Docker base image :

linuxbuild.dockerfile
1
2
3
4
FROM oracle/graalvm-ce:1.0.0-rc16

WORKDIR /tmp/dist
CMD native-image -jar /tmp/target/graalvm-scala-lambda.jar --enable-url-protocols=http bootstrap

The GraalVM Docker base image provides the Linux OS (oraclelinux, specifically) and the GraalVM tools. The input directory (/tmp/target) and output directory (/tmp/dist) will be mapped to local directories when the docker image is executed, enabling us to retrieve the generated executable. The --enable-url-protocols=http argument is required for http support by GraalVM which otherwise leaves it out to optimize the executable size.

A build script ties together SBT and the GraalVM build in Docker:

build.sh
1
2
3
4
5
6
7
8
9
#!/bin/sh
# Compiles to Java classes, builds a jar, and then compiles to a native Linux executable 

set -e

mkdir -p dist
sbt clean assembly
docker build -f linuxbuild.dockerfile -t linuxbuild .
docker run -v "$(pwd -P)/target/scala-2.12":/tmp/target -v "$(pwd -P)/dist":/tmp/dist linuxbuild

Mounting the target/scala-2.12 directory keeps the JAR file out of the docker image, which means you can build the docker once locally and simply run lines 7 and 9 while compiling new iterations of your code. Mounting the dist directory allows us to retrieve the executable file generated when the docker image is run, the entire point of this exercise.

Go ahead and run the build script to generate the Linux executable. Note that the GraalVM docker image is a hefty download so the first execution may take more than the standard 2 minutes to run.

$ ./build.sh 
[info] Loading project definition from /graalvm-scala-lambda/project
[info] Set current project to graalvm-scala-lambda (in build file:/graalvm-scala-lambda/)
...
[success] Total time: 14 s, completed May 22, 2019 9:05:06 AM
Sending build context to Docker daemon  38.79MB
..
Successfully tagged linuxbuild:latest
Build on Server(pid: 10, port: 42035)*
[bootstrap:10]    classlist:   9,268.85 ms
...
[bootstrap:10]      [total]:  60,824.79 ms

The output app, dist/bootstrap, is now ready to be tested locally with SAM.

About The SAM Configuration

Following the convention in the previous post I’m using SAM’s implicit endpoint configuration in template.yaml to define the AWS Lambda function and API Gateway endpoints togther. This has the same drawback as before, where the API Gateway StageName is set to Prod by SAM for implicit configurations, leading to a curious Prod mention in the endpoint path. If you prefer to keep production and non-production functions in separate AWS accounts, as I do, this is more an oddity than a problem.

Here is the full SAM configuration, which uses the custom runtime (“provided”) and points to the bootstrap executable in the dist local directory:

template.yaml 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
AWSTemplateFormatVersion: '2010-09-09'

Transform: AWS::Serverless-2016-10-31

Description: >-
  A Hello, World app for AWS Lambda. 
  Written in Scala, compiled with the GraalVM and executed with the AWS Lambda Custom Runtime.

Resources:
  HelloScalaGraalvmFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: HelloScalaGraalvm
      Description: Lambda function with custom runtime
      Runtime: provided
      Handler: bootstrap
      CodeUri: dist/
      MemorySize: 512
      Timeout: 15
      Events:
        Hello:
          Type: Api
          Properties:
            Path: /hello/{name}
            Method: GET
        
Outputs:
    ApiURL:
      Description: "API endpoint URL for Prod environment"
      Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/{name}"

The ApiURL output variable will generate the eventual url of the Lambda function when the CloudFormation stack is executed, allowing us to retrieve it with the AWS CLI.

Execution

Running The Function Locally

Now that you’ve run build.sh and generated a Linux executable, it’s time to try it out with the SAM CLI. Run the following command to start it up:

$ sam local start-api

Then, in a separate terminal, test it out with the curl command:

$ curl http://127.0.0.1:3000/hello/developer
Hello, developer

You should see the Hello, $name response from line 40 of src/main/scala/bootstrap.scala.

Now that the function is verified as working locally, it’s time to deploy it to AWS Lambda!

Deploying The Function To AWS

Create An S3 Bucket

To package and deploy your function to AWS you’ll need an S3 Bucket to hold the packaged (zipped) function code. Head over to the AWS S3 Console and create a new S3 bucket or select an existing S3 bucket to hold the code during deployment. Save the bucket name (just the name, not the full S3 URI) for the upcoming deploy step.

Upload Your Function

Run the following command to package and upload the function to your S3 bucket:

$ sam package --s3-bucket <your-s3-bucket-name> --output-template-file packaged.yaml

Pick An AWS Region

While S3 buckets are global, AWS Lambda functions and API Gateway endpoints are regional. Before deploying your function you will need to choose your AWS Region. When you have chosen a region (perhaps one close to you, or with a preferred pricing model) and its region code (eg “us-west-2”), run the following command to set your region for AWS operations:

$ export AWS_DEFAULT_REGION=<region-code>

Deploy Your Function

Run the following command to deploy your function from S3 to AWS Lambda in your selected region:

$ sam deploy --template-file packaged.yaml --stack-name hello-scala-graal --capabilities CAPABILITY_IAM

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - hello-scala-graal

After about a minute you should see the confirmation message that the CloudFormation stack “aws-lambda-hello-scala” has been created (and executed!). You now have an API Gateway endpoint and Lambda function deployed.

Retrieve the Endpoint URL

Run the following command to retrieve the rendered ApiURL output variable containing the Lambda’s endpoint (unnecessary details hidden):

$ aws cloudformation describe-stacks --stack-name hello-scala-graal
{
    "Stacks": [
        ...
        "StackName": "hello-scala-graal",
        ...
        "StackStatus": "CREATE_COMPLETE",
        ...
        "Outputs": [
            {
                "OutputKey": "ApiURL",
                "OutputValue": "https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/{name}",
                "Description": "API endpoint URL for Prod environment"
            }
        ],
        ...
      ]
  }

Test Your Deployed Function

Run the following command (using the actual hostname from the OutputValue above) to try out your deployed function and also measure its performance:

$ time curl https://f2c0a1aa55.execute-api.us-west-2.amazonaws.com/Prod/hello/developer

Hello, developer
curl   0.02s user 0.01s system 15% cpu 0.69s total

Now that the custom runtime has been verified as deployed and is warmed up (ie there is at least one instance of the function that is running in a continuous loop), run the command a few more times to verify the typical response time. You should see performance similar to these results:

$ time curl https://vwtd2iu524.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer!
curl   0.02s user 0.01s system 19% cpu 0.147 total
$ time curl https://vwtd2iu524.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer!
curl   0.02s user 0.01s system 15% cpu 0.186 total
$ time curl https://vwtd2iu524.execute-api.us-west-2.amazonaws.com/Prod/hello/developer
Hello, developer!
curl   0.02s user 0.01s system 19% cpu 0.153 total

Warmed up, the native function returns in an averaged 170 milliseconds.

Decommissioning Your Lambda Function

When you are finished with your function, you can decommission the API Gateway endpoint and AWS Lambda with the following CloudFormation CLI command:

$ aws cloudformation delete-stack --stack-name hello-scala-graal

After a few seconds you can verify the resources have been deleted by calling describe-stacks again:

$ aws cloudformation describe-stacks --stack-name hello-scala-graal

An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id hello-scala-graal does not exist

Optimizing Native Function Performance By Memory Allocation

In a recent post I write about measuring AWS Lambda function cold & warm response times for functions deployed with the Java runtime, which clearly benefits from increased memory sizes. I performed similar tests for the native function with the custom runtime covered in this post, with one improvement to “cold start” measurements. To better measure “cold start” times I added a sleep time of 6 minutes to the script and then accessed the function. I’m assuming this is a better test of the period after which AWS Lambda puts the runtime to “sleep” than the previous methodology of testing the function immediately after it is deployed, as “cold starts” are expected to occur after periods of inactivity.

Here’s the result of the response time per MemorySize measuring for this native function:

cold-warm-times

As you can see, other than the additional jump for the 256Mb setting, increase the allocated memory from 128Mb (minimum) to 3008Mb (maximum) had little effect on the response times. The “cold start” (ie, 6 minutes between accesses) response times averaged about 250ms while accesses separated by mere seconds averaged about 150ms.

Next Steps

Some additional areas to consider for performance gains may include:

  • Optimizing the code for reading and unmarshalling requests
  • Optimizing the code for marshalling and writing responses
  • Trying out different cross-compiling solutions such as Scala-Native or the upcoming production release of GraalVM
  • Using a natively-compiled language (eg C, Rust) for the network features and JSON handling

Conclusion

The GraalVM compiler together with the AWS Lambda custom runtime provide a high-performance solution for running Scala code in AWS Lambda than using the Java runtime. At 250ms for “cold starts” and 150ms for all other responses (albeit with an overly-simplified “Hello, World” function), this combination should provide an suitably performant platform for Scala functions in production.