API Gateway invoking Lambda function with Terraform - Lambda Container

In this post, we'll setup an API Gateway that invokes Lmabda function that takes an input. We'll do that via Terraform.

This post is similar to AWS API Gateway invoking Lambda function with Terraform, but this time, it will use ECR Docker image rather than using S3's zip file.

We'll be following the guidelines from:

1. Resource: aws_api_gateway_resource.
2. Serverless Applications with AWS Lambda and API Gateway
3. TERRAFORM – DEPLOY PYTHON LAMBDA (CONTAINER IMAGE)

To create a lambda using Docker image, we need the following files:

├── apps
│   ├── Dockerfile
│   ├── app.py
│   └── requirements.txt
├── lambda.tf
└── variables.tf    

Later, we'll add an API Gateway menifest file.

For now, let's see how the files look like.

apps/app.py:

def handler(event, context):

    h = float(event['hour'])
    print(h)

    return {
        'past hours': h
    }

As we can see, the app is doing a very simple thing: takes a vaule from query string and then just spits it.

apps/Dockerfile:

FROM public.ecr.aws/lambda/python:3.8
COPY requirements.txt ./
RUN yum update -y && \
    pip install -r requirements.txt
COPY app.py .
CMD ["app.handler"]

apps/requirements.tf:

requests==2.25.1

The "requests" module is not needed right now with our current code, but it shows how we install a package but just modifying the "requiremets.txt" rather than write a new Dockerfile. Actually, we'll later modify the app.py which makes some requests to a given url and then we need the "requests" module.

Now, move on to Terraform. Here is the lamba menifest file, lambda.tf:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

provider "aws" {
  region = var.region
}

data aws_caller_identity current {}
 
locals {
  prefix = "bogo"
  app_dir = "apps"
  account_id          = data.aws_caller_identity.current.account_id
  ecr_repository_name = "${local.prefix}-demo-lambda-container"
  ecr_image_tag       = "latest"
}
 
resource aws_ecr_repository repo {
  name = local.ecr_repository_name
}

# The null_resource resource implements the standard resource lifecycle 
# but takes no further action.

# The triggers argument allows specifying an arbitrary set of values that, 
# when changed, will cause the resource to be replaced.

resource null_resource ecr_image {
  triggers = {
    python_file = md5(file("${path.module}/${local.app_dir}/app.py"))
    docker_file = md5(file("${path.module}/${local.app_dir}/Dockerfile"))
}
 
# The local-exec provisioner invokes a local executable after a resource is created. 
# This invokes a process on the machine running Terraform, not on the resource. 
# path.module: the filesystem path of the module where the expression is placed.

provisioner "local-exec" {
  command = <<EOF
           aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
           cd ${path.module}/${local.app_dir}
           docker build -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
           docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
       EOF
 }
}

data aws_ecr_image lambda_image {
  depends_on = [
    null_resource.ecr_image
  ]
  repository_name = local.ecr_repository_name
  image_tag       = local.ecr_image_tag
}

resource aws_iam_role lambda {
  name = "${local.prefix}-lambda-role"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Action": "sts:AssumeRole",
          "Principal": {
              "Service": "lambda.amazonaws.com"
          },
          "Effect": "Allow"
      }
  ]
}
  EOF
}

data aws_iam_policy_document lambda {
  statement {
    actions = [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
    ]
    effect = "Allow"
    resources = [ "*" ]
    sid = "CreateCloudWatchLogs"
  }
 
  statement {
    actions = [
        "codecommit:GitPull",
        "codecommit:GitPush",
        "codecommit:GitBranch",
        "codecommit:ListBranches",
        "codecommit:CreateCommit",
        "codecommit:GetCommit",
        "codecommit:GetCommitHistory",
        "codecommit:GetDifferences",
        "codecommit:GetReferences",
        "codecommit:BatchGetCommits",
        "codecommit:GetTree",
        "codecommit:GetObjectIdentifier",
        "codecommit:GetMergeCommit"
    ]
    effect = "Allow"
    resources = [ "*" ]
    sid = "CodeCommit"
  }
}

resource aws_iam_policy lambda {
  name = "${local.prefix}-lambda-policy"
  path = "/"
  policy = data.aws_iam_policy_document.lambda.json
}

resource aws_lambda_function bogo-lambda-function {
  depends_on = [
    null_resource.ecr_image
  ]
  function_name = "${local.prefix}-lambda"
  role = aws_iam_role.lambda.arn
  timeout = 300
  image_uri = "${aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id}"
  package_type = "Image"
}
 
output "lambda_name" {
  value = aws_lambda_function.bogo-lambda-function.id
}

The file will create an ECR repository, build a Docker image, and then upload it to the repo.
It also creates a lambda function using the container image.

In the process, it creates a policy and role to do those tasks.

The variables used in the file is defined in vaiables.tf:

variable region {
 default = "us-east-1"
}

variable "app_version" {
    default = "1.0.0"
}

variable "stage" {
	default = "dev"
}

variable "resource_name" {
	default = "number"
}   

Now, we are ready to deploy our lambda:

$ terraform init

$ terraform apply--auto-approve
Outputs:

lambda_name = "bogo-lambda"

Our lambda function that uses the Docker image has been created:

We may want to test it:

Here is our manifest file for AWS API Gateway, api-gateway.tf:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

provider "aws" {
  region = var.region
}

data aws_caller_identity current {}
 
locals {
  prefix = "bogo"
  app_dir = "apps"
  account_id          = data.aws_caller_identity.current.account_id
  ecr_repository_name = "${local.prefix}-demo-lambda-container"
  ecr_image_tag       = "latest"
}
 
resource aws_ecr_repository repo {
  name = local.ecr_repository_name
}

# The null_resource resource implements the standard resource lifecycle 
# but takes no further action.

# The triggers argument allows specifying an arbitrary set of values that, 
# when changed, will cause the resource to be replaced.

resource null_resource ecr_image {
  triggers = {
    python_file = md5(file("${path.module}/${local.app_dir}/app.py"))
    docker_file = md5(file("${path.module}/${local.app_dir}/Dockerfile"))
}
 
# The local-exec provisioner invokes a local executable after a resource is created. 
# This invokes a process on the machine running Terraform, not on the resource. 
# path.module: the filesystem path of the module where the expression is placed.

provisioner "local-exec" {
  command = <<EOF
           aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
           #cd ${path.module}/lambdas/${local.prefix}_client
           cd ${path.module}/${local.app_dir}
           docker build -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
           docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
       EOF
 }
}

data aws_ecr_image lambda_image {
  depends_on = [
    null_resource.ecr_image
  ]
  repository_name = local.ecr_repository_name
  image_tag       = local.ecr_image_tag
}

resource aws_iam_role lambda {
  name = "${local.prefix}-lambda-role"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Action": "sts:AssumeRole",
          "Principal": {
              "Service": "lambda.amazonaws.com"
          },
          "Effect": "Allow"
      }
  ]
}
  EOF
}

data aws_iam_policy_document lambda {
  statement {
    actions = [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
    ]
    effect = "Allow"
    resources = [ "*" ]
    sid = "CreateCloudWatchLogs"
  }
 
  statement {
    actions = [
        "codecommit:GitPull",
        "codecommit:GitPush",
        "codecommit:GitBranch",
        "codecommit:ListBranches",
        "codecommit:CreateCommit",
        "codecommit:GetCommit",
        "codecommit:GetCommitHistory",
        "codecommit:GetDifferences",
        "codecommit:GetReferences",
        "codecommit:BatchGetCommits",
        "codecommit:GetTree",
        "codecommit:GetObjectIdentifier",
        "codecommit:GetMergeCommit"
    ]
    effect = "Allow"
    resources = [ "*" ]
    sid = "CodeCommit"
  }
}

resource aws_iam_policy lambda {
  name = "${local.prefix}-lambda-policy"
  path = "/"
  policy = data.aws_iam_policy_document.lambda.json
}

resource aws_lambda_function bogo-lambda-function {
  depends_on = [
    null_resource.ecr_image
  ]
  function_name = "${local.prefix}-lambda"
  role = aws_iam_role.lambda.arn
  timeout = 300
  image_uri = "${aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id}"
  package_type = "Image"
}

resource "aws_lambda_permission" "apigw" {
   statement_id  = "AllowAPIGatewayInvoke"
   action        = "lambda:InvokeFunction"
   function_name = aws_lambda_function.bogo-lambda-function.function_name
   principal     = "apigateway.amazonaws.com"

   # The "/*/*" portion grants access from any method on any resource
   # within the API Gateway REST API.
   source_arn = "${aws_api_gateway_rest_api.bogo-gateway.execution_arn}/*/*"
}

 
output "lambda_name" {
  value = aws_lambda_function.bogo-lambda-function.id
}

We need to let the lambda function that the trigger is the API Gateway. So, here is our final lambda.tf file:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

provider "aws" {
  region = var.region
}

data aws_caller_identity current {}
 
locals {
  prefix = "bogo"
  app_dir = "apps"
  account_id          = data.aws_caller_identity.current.account_id
  ecr_repository_name = "${local.prefix}-demo-lambda-container"
  ecr_image_tag       = "latest"
}
 
resource aws_ecr_repository repo {
  name = local.ecr_repository_name
}

# The null_resource resource implements the standard resource lifecycle 
# but takes no further action.

# The triggers argument allows specifying an arbitrary set of values that, 
# when changed, will cause the resource to be replaced.

resource null_resource ecr_image {
  triggers = {
    python_file = md5(file("${path.module}/${local.app_dir}/app.py"))
    docker_file = md5(file("${path.module}/${local.app_dir}/Dockerfile"))
}
 
# The local-exec provisioner invokes a local executable after a resource is created. 
# This invokes a process on the machine running Terraform, not on the resource. 
# path.module: the filesystem path of the module where the expression is placed.

provisioner "local-exec" {
  command = <<EOF
           aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
           cd ${path.module}/${local.app_dir}
           docker build -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
           docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
       EOF
 }
}

data aws_ecr_image lambda_image {
  depends_on = [
    null_resource.ecr_image
  ]
  repository_name = local.ecr_repository_name
  image_tag       = local.ecr_image_tag
}

resource aws_iam_role lambda {
  name = "${local.prefix}-lambda-role"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Action": "sts:AssumeRole",
          "Principal": {
              "Service": "lambda.amazonaws.com"
          },
          "Effect": "Allow"
      }
  ]
}
  EOF
}

data aws_iam_policy_document lambda {
  statement {
    actions = [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
    ]
    effect = "Allow"
    resources = [ "*" ]
    sid = "CreateCloudWatchLogs"
  }
 
  statement {
    actions = [
        "codecommit:GitPull",
        "codecommit:GitPush",
        "codecommit:GitBranch",
        "codecommit:ListBranches",
        "codecommit:CreateCommit",
        "codecommit:GetCommit",
        "codecommit:GetCommitHistory",
        "codecommit:GetDifferences",
        "codecommit:GetReferences",
        "codecommit:BatchGetCommits",
        "codecommit:GetTree",
        "codecommit:GetObjectIdentifier",
        "codecommit:GetMergeCommit"
    ]
    effect = "Allow"
    resources = [ "*" ]
    sid = "CodeCommit"
  }
}

resource aws_iam_policy lambda {
  name = "${local.prefix}-lambda-policy"
  path = "/"
  policy = data.aws_iam_policy_document.lambda.json
}

resource aws_lambda_function bogo-lambda-function {
  depends_on = [
    null_resource.ecr_image
  ]
  function_name = "${local.prefix}-lambda"
  role = aws_iam_role.lambda.arn
  timeout = 300
  image_uri = "${aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id}"
  package_type = "Image"
}

resource "aws_lambda_permission" "apigw" {
   statement_id  = "AllowAPIGatewayInvoke"
   action        = "lambda:InvokeFunction"
   function_name = aws_lambda_function.bogo-lambda-function.function_name
   principal     = "apigateway.amazonaws.com"

   # The "/*/*" portion grants access from any method on any resource
   # within the API Gateway REST API.
   source_arn = "${aws_api_gateway_rest_api.bogo-gateway.execution_arn}/*/*"
}
 
output "lambda_name" {
  value = aws_lambda_function.bogo-lambda-function.id
}    

Here are the final files:

.
├── api-gateway.tf
├── apps
│   ├── Dockerfile
│   ├── app.py
│   └── requirements.txt
├── lambda.tf
└── variables.tf    

The files are avaialble from API-Gateway-invoking-Lambda-function-with-Terraform-Lambda-Container

Now, we are ready to deploy our modified Terraform codes:

$ terraform apply --auto-approve
...
Outputs:

base_url = "https://jgyexabhp0.execute-api.us-east-1.amazonaws.com/dev/number"
lambda_name = "bogo-lambda"    

To check if our API GateWay triggers our lambda function correctly:

$ curl https://jgyexabhp0.execute-api.us-east-1.amazonaws.com/dev/number?hour=999
{"past hours": 999.0}    

It's working!

Here is our newly created with a API Gateway trigger:

Here is screenshot for our API Gateway:

AWS (Amazon Web Services)

1. AWS : EKS (Elastic Container Service for Kubernetes)
2. AWS : Creating a snapshot (cloning an image)
3. AWS : Attaching Amazon EBS volume to an instance
4. AWS : Adding swap space to an attached volume via mkswap and swapon
5. AWS : Creating an EC2 instance and attaching Amazon EBS volume to the instance using Python boto module with User data
6. AWS : Creating an instance to a new region by copying an AMI
7. AWS : S3 (Simple Storage Service) 1
8. AWS : S3 (Simple Storage Service) 2 - Creating and Deleting a Bucket
9. AWS : S3 (Simple Storage Service) 3 - Bucket Versioning
10. AWS : S3 (Simple Storage Service) 4 - Uploading a large file
11. AWS : S3 (Simple Storage Service) 5 - Uploading folders/files recursively
12. AWS : S3 (Simple Storage Service) 6 - Bucket Policy for File/Folder View/Download
13. AWS : S3 (Simple Storage Service) 7 - How to Copy or Move Objects from one region to another
14. AWS : S3 (Simple Storage Service) 8 - Archiving S3 Data to Glacier
15. AWS : Creating a CloudFront distribution with an Amazon S3 origin
16. AWS : Creating VPC with CloudFormation
17. AWS : WAF (Web Application Firewall) with preconfigured CloudFormation template and Web ACL for CloudFront distribution
18. AWS : CloudWatch & Logs with Lambda Function / S3
19. AWS : Lambda Serverless Computing with EC2, CloudWatch Alarm, SNS
20. AWS : Lambda and SNS - cross account
21. AWS : CLI (Command Line Interface)
22. AWS : CLI (ECS with ALB & autoscaling)
23. AWS : ECS with cloudformation and json task definition
24. AWS Application Load Balancer (ALB) and ECS with Flask app
25. AWS : Load Balancing with HAProxy (High Availability Proxy)
26. AWS : VirtualBox on EC2
27. AWS : NTP setup on EC2
28. AWS: jq with AWS
29. AWS & OpenSSL : Creating / Installing a Server SSL Certificate
30. AWS : OpenVPN Access Server 2 Install
31. AWS : VPC (Virtual Private Cloud) 1 - netmask, subnets, default gateway, and CIDR
32. AWS : VPC (Virtual Private Cloud) 2 - VPC Wizard
33. AWS : VPC (Virtual Private Cloud) 3 - VPC Wizard with NAT
34. DevOps / Sys Admin Q & A (VI) - AWS VPC setup (public/private subnets with NAT)
35. AWS - OpenVPN Protocols : PPTP, L2TP/IPsec, and OpenVPN
36. AWS : Autoscaling group (ASG)
37. AWS : Setting up Autoscaling Alarms and Notifications via CLI and Cloudformation
38. AWS : Adding a SSH User Account on Linux Instance
39. AWS : Windows Servers - Remote Desktop Connections using RDP
40. AWS : Scheduled stopping and starting an instance - python & cron
41. AWS : Detecting stopped instance and sending an alert email using Mandrill smtp
42. AWS : Elastic Beanstalk with NodeJS
43. AWS : Elastic Beanstalk Inplace/Rolling Blue/Green Deploy
44. AWS : Identity and Access Management (IAM) Roles for Amazon EC2
45. AWS : Identity and Access Management (IAM) Policies, sts AssumeRole, and delegate access across AWS accounts
46. AWS : Identity and Access Management (IAM) sts assume role via aws cli2
47. AWS : Creating IAM Roles and associating them with EC2 Instances in CloudFormation
48. AWS Identity and Access Management (IAM) Roles, SSO(Single Sign On), SAML(Security Assertion Markup Language), IdP(identity provider), STS(Security Token Service), and ADFS(Active Directory Federation Services)
49. AWS : Amazon Route 53
50. AWS : Amazon Route 53 - DNS (Domain Name Server) setup
51. AWS : Amazon Route 53 - subdomain setup and virtual host on Nginx
52. AWS Amazon Route 53 : Private Hosted Zone
53. AWS : SNS (Simple Notification Service) example with ELB and CloudWatch
54. AWS : Lambda with AWS CloudTrail
55. AWS : SQS (Simple Queue Service) with NodeJS and AWS SDK
56. AWS : Redshift data warehouse
57. AWS : CloudFormation
58. AWS : CloudFormation Bootstrap UserData/Metadata
59. AWS : CloudFormation - Creating an ASG with rolling update
60. AWS : Cloudformation Cross-stack reference
61. AWS : OpsWorks
62. AWS : Network Load Balancer (NLB) with Autoscaling group (ASG)
63. AWS CodeDeploy : Deploy an Application from GitHub
64. AWS EC2 Container Service (ECS)
65. AWS EC2 Container Service (ECS) II
66. AWS Hello World Lambda Function
67. AWS Lambda Function Q & A
68. AWS Node.js Lambda Function & API Gateway
69. AWS API Gateway endpoint invoking Lambda function
70. AWS API Gateway invoking Lambda function with Terraform
71. AWS API Gateway invoking Lambda function with Terraform - Lambda Container
72. Amazon Kinesis Streams
73. AWS: Kinesis Data Firehose with Lambda and ElasticSearch
74. Amazon DynamoDB
75. Amazon DynamoDB with Lambda and CloudWatch
76. Loading DynamoDB stream to AWS Elasticsearch service with Lambda
77. Amazon ML (Machine Learning)
78. Simple Systems Manager (SSM)
79. AWS : RDS Connecting to a DB Instance Running the SQL Server Database Engine
80. AWS : RDS Importing and Exporting SQL Server Data
81. AWS : RDS PostgreSQL & pgAdmin III
82. AWS : RDS PostgreSQL 2 - Creating/Deleting a Table
83. AWS : MySQL Replication : Master-slave
84. AWS : MySQL backup & restore
85. AWS RDS : Cross-Region Read Replicas for MySQL and Snapshots for PostgreSQL
86. AWS : Restoring Postgres on EC2 instance from S3 backup
87. AWS : Q & A
88. AWS : Security
89. AWS : Security groups vs. network ACLs
90. AWS : Scaling-Up
91. AWS : Networking
92. AWS : Single Sign-on (SSO) with Okta
93. AWS : JIT (Just-in-Time) with Okta