diff --git a/lambda-durable-function-chaining-terraform/README.md b/lambda-durable-function-chaining-terraform/README.md new file mode 100644 index 0000000000..afbafe1e96 --- /dev/null +++ b/lambda-durable-function-chaining-terraform/README.md @@ -0,0 +1,105 @@ +# Function chaining with AWS Lambda durable functions in Terraform + +This Terraform pattern demonstrates function chaining using AWS Lambda durable functions. A durable orchestrator invokes three Lambda functions in sequence (add, transform and finalize). The output of each step is passed as the input to the next. The framework automatically checkpoints after every step, so if the orchestrator fails mid-workflow, it replays from the beginning and skips already-completed work. This ensures exactly-once execution without re-processing. + +Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/lambda-durable-function-chaining-terraform + +Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. + +## Requirements + +* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. +* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured +* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +* [Terraform](https://learn.hashicorp.cxom/tutorials/terraform/install-cli?in=terraform/aws-get-started) installed + +## Deployment Instructions + +1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: + ``` + git clone https://github.com/aws-samples/serverless-patterns + ``` +1. Change directory to the pattern directory: + ``` + cd lambda-durable-function-chaining-terraform + ``` +1. From the command line, initialize terraform to downloads and installs the providers defined in the configuration: + ``` + terraform init + ``` +1. From the command line, apply the configuration in the main.tf file: + ``` + terraform apply -auto-approve + ``` +1. During the prompts: + #var.aws_region + - Enter a value: {enter the region for deployment} + + #var.prefix + - Enter a value: {enter any prefix to associate with resources} + +1. Note the outputs from the Terraform deployment process. These contain the resource names and/or ARNs which are used for testing. + +## How it works + +![architecture](architecture/architecture.png) + +This pattern uses a durable orchestrator to chain three Lambda functions in sequence. Each function receives the output of the previous step as its input, forming a pipeline: **Add → Transform → Finalize**. + +At each `context.invoke()` call, the durable framework creates a checkpoint. If the orchestrator fails at any point, it automatically replays from the beginning but skips already-completed steps using stored results, ensuring no work is repeated. Workflows resume from the last successful checkpoint, recovering automatically without manual intervention. + +Consider the following input, + +``` +Input: {"id": "test-1", "name": "demo", "value": 10} + +Step 1 (Add): value = 10 + 10 = 20 +Step 2 (Transform): value = 20 × 2 = 40, name = "DEMO" +Step 3 (Finalize): value = 40 + 5 = 45, status = "COMPLETED" + +Output: {"id": "test-123", "name": "demo", "step1_completed": true, "step1_value": 20, "step2_completed": true, "step2_value": 40, "transformed_name": "DEMO", "step3_completed": true, "status": "COMPLETED", "final_value": 45} +``` + +## Testing + +1. You can invoke the Lambda function by using the following CLI command: + ``` + aws lambda invoke --function-name "ORCHESTRATOR_ALIAS_ARN" --payload '{"id": "test-123", "name": "demo", "value": 10}' --cli-binary-format raw-in-base64-out response.json + ``` + Note: Replace the `ORCHESTRATOR_ALIAS_ARN` with the generated `orchestrator_alias-arn` from Terraform (refer to the Terraform Outputs section) + +1. Display the contents of response.json + ``` + cat response.json + ``` + + Expected output + ``` + {"id": "test-123", "name": "demo", "step1_completed": true, "step1_value": 20, "step2_completed": true, "step2_value": 40, "transformed_name": "DEMO", "step3_completed": true, "status": "COMPLETED", "final_value": 45} + ``` + +## Cleanup + +1. Change directory to the pattern directory: + ``` + cd serverless-patterns/lambda-durable-function-chaining-terraform + ``` + +1. Delete all created resources + ``` + terraform destroy -auto-approve + ``` + +1. During the prompts: + ``` + Enter all details as entered during creation. + ``` + +1. Confirm all created resources has been deleted + ``` + terraform show + ``` +---- +Copyright 2026 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 \ No newline at end of file diff --git a/lambda-durable-function-chaining-terraform/architecture/architecture.png b/lambda-durable-function-chaining-terraform/architecture/architecture.png new file mode 100644 index 0000000000..42667c1af2 Binary files /dev/null and b/lambda-durable-function-chaining-terraform/architecture/architecture.png differ diff --git a/lambda-durable-function-chaining-terraform/example-pattern.json b/lambda-durable-function-chaining-terraform/example-pattern.json new file mode 100644 index 0000000000..1af5adfb6c --- /dev/null +++ b/lambda-durable-function-chaining-terraform/example-pattern.json @@ -0,0 +1,58 @@ +{ + "title": "Function chaining with AWS Lambda durable functions in Terraform", + "description": "A durable orchestrator that chains three Lambda functions in sequence to demonstrate AWS Lambda's durable function", + "language": "Python", + "level": "300", + "framework": "Terraform", + "introBox": { + "headline": "How it works", + "text": [ + "A durable orchestrator invokes three Lambda functions in sequence, passing each step's output as input to the next. The durable framework checkpoints after every `context.invoke()` call and if the orchestrator fails, it replays from the beginning but skips completed steps using stored results, ensuring no work is repeated." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-function-chaining-terraform", + "templateURL": "serverless-patterns/lambda-durable-function-chaining-terraform", + "projectFolder": "lambda-durable-function-chaining-terraform", + "templateFile": "main.tf" + } + }, + "resources": { + "bullets": [ + { + "text": "Lambda durable functions", + "link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html" + }, + { + "text": "Build multi-step applications and AI workflows with AWS Lambda durable functions", + "link": "https://aws.amazon.com/blogs/aws/build-multi-step-applications-and-ai-workflows-with-aws-lambda-durable-functions/" + } + ] + }, + "deploy": { + "text": [ + "terraform init", + "terraform apply" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "terraform destroy", + "terraform show" + ] + }, + "authors": [ + { + "name": "Archana V", + "image": "ttps://media.licdn.com/dms/image/v2/D5603AQGhkVtEhllFEw/profile-displayphoto-shrink_200_200/B56ZZH3LL6H0AY-/0/1744962369852?e=1772668800&v=beta&t=y0t7bsQKJwy5McO395hDmW7QPu_K-a1WKXeTA0-ecno", + "bio": "Solutions Architect at AWS", + "linkedin": "archanavenkat" + } + ] +} diff --git a/lambda-durable-function-chaining-terraform/main.tf b/lambda-durable-function-chaining-terraform/main.tf new file mode 100644 index 0000000000..bcfa6aa153 --- /dev/null +++ b/lambda-durable-function-chaining-terraform/main.tf @@ -0,0 +1,303 @@ +############################################################ +# Terraform Configuration +############################################################ + +terraform { + required_version = ">= 1.0" + + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 6.32.1" + } + } +} + +############################################################ +# Provider +############################################################ + +provider "aws" { + region = var.aws_region +} + +############################################################ +# Variables +############################################################ + +variable "aws_region" { + description = "AWS region for resources (e.g. us-east-1, us-west-2)" + type = string + + validation { + condition = can(regex("^[a-z]{2}-[a-z]+-[0-9]+$", var.aws_region)) + error_message = "Must be a valid AWS region (e.g. us-east-1, eu-west-2)." + } +} + +variable "prefix" { + description = "Unique prefix for all resource names" + type = string + + validation { + condition = can(regex("^[a-z0-9][a-z0-9\\-]{1,20}$", var.prefix)) + error_message = "Prefix must be 2-21 lowercase alphanumeric characters or hyphens." + } +} + +variable "log_retention_days" { + description = "CloudWatch log retention in days (0 = never expire)" + type = number + default = 14 +} + +############################################################ +# IAM Role for All Lambda Functions +############################################################ + +resource "aws_iam_role" "lambda_role" { + name = "${var.prefix}-durable-orchestrator-role" + + force_detach_policies = true + + assume_role_policy = jsonencode({ + Version = "2012-10-17" + Statement = [{ + Effect = "Allow" + Action = "sts:AssumeRole" + Principal = { Service = "lambda.amazonaws.com" } + }] + }) +} + +resource "aws_iam_role_policy_attachment" "basic_execution" { + role = aws_iam_role.lambda_role.name + policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" +} + +resource "aws_iam_role_policy_attachment" "durable_execution" { + role = aws_iam_role.lambda_role.name + policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicDurableExecutionRolePolicy" +} + +############################################################ +# CloudWatch Log Groups (Terraform-managed) +############################################################ + +resource "aws_cloudwatch_log_group" "step1" { + name = "/aws/lambda/${var.prefix}-step1-add" + retention_in_days = var.log_retention_days + skip_destroy = false +} + +resource "aws_cloudwatch_log_group" "step2" { + name = "/aws/lambda/${var.prefix}-step2-transform" + retention_in_days = var.log_retention_days + skip_destroy = false +} + +resource "aws_cloudwatch_log_group" "step3" { + name = "/aws/lambda/${var.prefix}-step3-finalize" + retention_in_days = var.log_retention_days + skip_destroy = false +} + +resource "aws_cloudwatch_log_group" "orchestrator" { + name = "/aws/lambda/${var.prefix}-durable-orchestrator" + retention_in_days = var.log_retention_days + skip_destroy = false +} + +############################################################ +# Scoped CloudWatch Logs Write Policy +############################################################ + +resource "aws_iam_role_policy" "lambda_logging" { + name = "${var.prefix}-lambda-cloudwatch-logging" + role = aws_iam_role.lambda_role.id + + policy = jsonencode({ + Version = "2012-10-17" + Statement = [{ + Effect = "Allow" + Action = [ + "logs:CreateLogStream", + "logs:PutLogEvents" + ] + Resource = [ + "${aws_cloudwatch_log_group.step1.arn}:*", + "${aws_cloudwatch_log_group.step2.arn}:*", + "${aws_cloudwatch_log_group.step3.arn}:*", + "${aws_cloudwatch_log_group.orchestrator.arn}:*" + ] + }] + }) +} + +############################################################ +# Worker Lambda Functions +############################################################ + +resource "aws_lambda_function" "step1" { + function_name = "${var.prefix}-step1-add" + filename = "step1_add.zip" + handler = "step1_add.lambda_handler" + runtime = "python3.14" + memory_size = 512 + timeout = 30 + role = aws_iam_role.lambda_role.arn + + logging_config { + log_group = aws_cloudwatch_log_group.step1.name + log_format = "Text" + } + + depends_on = [ + aws_cloudwatch_log_group.step1, + aws_iam_role_policy.lambda_logging + ] +} + +resource "aws_lambda_function" "step2" { + function_name = "${var.prefix}-step2-transform" + filename = "step2_transform.zip" + handler = "step2_transform.lambda_handler" + runtime = "python3.14" + memory_size = 512 + timeout = 30 + role = aws_iam_role.lambda_role.arn + + logging_config { + log_group = aws_cloudwatch_log_group.step2.name + log_format = "Text" + } + + depends_on = [ + aws_cloudwatch_log_group.step2, + aws_iam_role_policy.lambda_logging + ] +} + +resource "aws_lambda_function" "step3" { + function_name = "${var.prefix}-step3-finalize" + filename = "step3_finalize.zip" + handler = "step3_finalize.lambda_handler" + runtime = "python3.14" + memory_size = 512 + timeout = 30 + role = aws_iam_role.lambda_role.arn + + logging_config { + log_group = aws_cloudwatch_log_group.step3.name + log_format = "Text" + } + + depends_on = [ + aws_cloudwatch_log_group.step3, + aws_iam_role_policy.lambda_logging + ] +} + +############################################################ +# Allow Orchestrator to Invoke Worker Functions +############################################################ + +resource "aws_iam_role_policy" "invoke_workers" { + name = "${var.prefix}-invoke-worker-functions" + role = aws_iam_role.lambda_role.id + + policy = jsonencode({ + Version = "2012-10-17" + Statement = [{ + Effect = "Allow" + Action = ["lambda:InvokeFunction"] + Resource = [ + aws_lambda_function.step1.arn, + aws_lambda_function.step2.arn, + aws_lambda_function.step3.arn, + "${aws_lambda_function.step1.arn}:*", + "${aws_lambda_function.step2.arn}:*", + "${aws_lambda_function.step3.arn}:*" + ] + }] + }) +} + +############################################################ +# Durable Orchestrator Function +############################################################ + +resource "aws_lambda_function" "orchestrator" { + function_name = "${var.prefix}-durable-orchestrator" + filename = "orchestrator.zip" + handler = "orchestrator.lambda_handler" + runtime = "python3.14" + memory_size = 512 + timeout = 90 + role = aws_iam_role.lambda_role.arn + publish = true + + durable_config { + execution_timeout = 90 + retention_period = 7 + } + + logging_config { + log_group = aws_cloudwatch_log_group.orchestrator.name + log_format = "JSON" + } + + environment { + variables = { + STEP1_FUNCTION_ARN = aws_lambda_function.step1.arn + STEP2_FUNCTION_ARN = aws_lambda_function.step2.arn + STEP3_FUNCTION_ARN = aws_lambda_function.step3.arn + } + } + + timeouts { + delete = "6m" + } + + tags = { + Project = "${var.prefix}-durable-workflow" + Environment = "dev" + Type = "durable" + } + + depends_on = [ + aws_cloudwatch_log_group.orchestrator, + aws_iam_role_policy.lambda_logging + ] +} + +############################################################ +# Alias +############################################################ + +resource "aws_lambda_alias" "live" { + name = "live" + function_name = aws_lambda_function.orchestrator.function_name + function_version = aws_lambda_function.orchestrator.version +} + +############################################################ +# Outputs +############################################################ + +output "orchestrator_alias_arn" { + description = "Invoke this ARN to start the durable workflow" + value = aws_lambda_alias.live.arn +} + +output "step1_function_arn" { + value = aws_lambda_function.step1.arn +} + +output "step2_function_arn" { + value = aws_lambda_function.step2.arn +} + +output "step3_function_arn" { + value = aws_lambda_function.step3.arn +} \ No newline at end of file diff --git a/lambda-durable-function-chaining-terraform/orchestrator.zip b/lambda-durable-function-chaining-terraform/orchestrator.zip new file mode 100644 index 0000000000..a5209be2fd Binary files /dev/null and b/lambda-durable-function-chaining-terraform/orchestrator.zip differ diff --git a/lambda-durable-function-chaining-terraform/step1_add.zip b/lambda-durable-function-chaining-terraform/step1_add.zip new file mode 100644 index 0000000000..20b6bf3dac Binary files /dev/null and b/lambda-durable-function-chaining-terraform/step1_add.zip differ diff --git a/lambda-durable-function-chaining-terraform/step2_transform.zip b/lambda-durable-function-chaining-terraform/step2_transform.zip new file mode 100644 index 0000000000..db09f8951e Binary files /dev/null and b/lambda-durable-function-chaining-terraform/step2_transform.zip differ diff --git a/lambda-durable-function-chaining-terraform/step3_finalize.zip b/lambda-durable-function-chaining-terraform/step3_finalize.zip new file mode 100644 index 0000000000..1e66cab31b Binary files /dev/null and b/lambda-durable-function-chaining-terraform/step3_finalize.zip differ