We're big fans of Amazon Web Services. While we run our heaviest workloads directly on EC2 for cost efficiency reasons, we've adopted AWS API Gateway and AWS Lambda for on-demand workloads where flexibility is important. It's hard to get started, so we're sharing a template we use. It shows how to use CloudFormation to publish a REST API using:
Just want the code? You can skip to the associated repository on GitHub, with Java and Nodejs 10.x examples.
We're going to build an end-to-end service that echoes the user's input. You could use any of Lambda's supported languages, but we'll use Java . When everything's done, you'll be able to send a request like this:
$ curl https://echo.code402.com/echo?message=hello HELLO
We could do this entirely through the AWS UI, but instead we'd like to describe the infrastructure with a CloudFormation template. This delivers some real benefits: teammates can code review the infrastructure, developers can easily duplicate the stack to deploy a staging environment, and the ops team can easily replicate the stack to multiple regions for availability or performance improvements. We use CloudFormation because we like to stick to AWS-specific tooling, but you may also want to consider competing tools like Terraform or Serverless.
Since the actual service isn't too important, we'll write one that receives some text and responds with SOME TEXT. Say hello to the Shouty Echo Service! It's quite short:
Even with no prior experience with API Gateway, you can likely see what's going on. The APIGatewayProxyRequestEvent contains the properties of the HTTP request. Your service's response is written to the APIGatewayProxyResponseEvent object .
The service and its dependencies need to be packaged into a single, self-contained uberjar. Our GitHub repository includes a sample project that does just that.
CloudFormation translates a text file describing AWS resources into live instances of those resources. This "stack" of resources can be monitored using the AWS CloudFormation console. The text file itself can be YAML or JSON. Most people use YAML, since it supports comments.
We'll construct our template step-by-step from the ground up. First we'll create the AWS Lambda that runs the shouty echo service. Then we'll connect it to CloudWatch for logging. Next, we'll connect it to API Gateway so that anonymous Internet users can invoke it over HTTP. Finally, we'll integrate with Route 53 and AWS Certificate Manager to give it a custom domain name.
Start by provisioning a Lambda function that runs the service:
This template introduces two high-level CloudFormation concepts:
If you have previous experience configuring Lambda functions, you'll see that this template describes the steps you would have otherwise taken in the UI. The !Ref and !GetAtt functions are intrinsic functions that CloudFormation provides. They're needed when a hardcoded value isn't enough, for example, to include a user-provided parameter value, or to reference the Amazon Resource Name of a resource that will be created as part of the stack.
You could provision this stack immediately through the UI, or via the command-line: aws cloudformation create-stack --region us-east-1 --stack-name lambda-demo --template-body file://stack-lambda.yaml --capabilities CAPABILITY_IAM --parameters ParameterKey=S3Bucket,ParameterValue=your-bucket-name
To ease debugging, we'd like to log output from the Lambda to CloudWatch. By default, Lambda tries to log to a CloudWatch log group named after the function name of the Lambda. In order for it to work, we need to update the IAM role's permissions to include a policy that grants it access to that log group:
Logs default to being kept forever. That adds up, so we'll explicitly limit them to 14 days:
Note that we've introduced the Sub intrinsic function to construct strings that contain your account ID, AWS region and Lambda function name.
To expose this service to the public Internet, we turn to API Gateway. There's a lot going on, so we'll use inline comments to break it down:
Lastly, we introduce a new top-level section, the Outputs section. Because CloudFormation discourages hardcoding values, the identifiers of the resources will change each time you rebuild the stack. That's a pain! Outputs let you collect those values from each run in a central place. They show up in the UI like this:
The shouty echo service is now available!
$ curl https://6jdt0ybq1e.execute-api.us-east-1.amazonaws.com/echo?message=hello HELLO
API Gateway domain names are hard to remember, so we'll configure a vanity subdomain, for example echo.code402.com. Since we host our domain with Route 53, this is almost painless! We'll just have to add some entries to our CloudFormation file. First, we'll add some parameters to capture the desired domain name and the ID of the Route 53 zone:
Next, we'll add some new resources to register the domain name, request an SSL certificate from AWS Certificate Manager, and tell API Gateway to respond on the domain name:
There's a weird quirk here. Before it issues an SSL certificate, AWS Certificate Manager requires proof of ownership of the domain name. You'd think that since we're using Route 53 to manage the domain, AWS could verify this without our assistance, but it turns out it can't. Instead, when we apply the template, CloudFormation pauses until we create the verification record ourself in Route 53:
Once the record is created, CloudFormation will continue provisioning the stack. When it's all done, we can do:
$ curl https://echo.code402.com/echo?message=hello HELLO
We think that CloudFormation, while initially daunting, unlocks some real value. The template file we built can be peer-reviewed by other members of the team. It can be checked in to source control so future edits are subject to review. A developer new to the project can spin up their own entirely self-contained stack to use as a sandbox for testing.