From c8075d187bf48d1e1f51327738decebcbe28821c Mon Sep 17 00:00:00 2001 From: 4D54 <78043109+4D54@users.noreply.github.com> Date: Thu, 5 Feb 2026 19:46:42 +0100 Subject: [PATCH 1/2] Add Step Functions to ECS Python integration pattern - Demonstrates synchronous (polling) and callback integration patterns - Includes inline Python code in ECS task definitions - Complete SAM template with VPC, networking, and IAM roles - Comprehensive documentation and testing instructions --- sfn-ecs-python-sam/README.md | 349 ++++++++++++++++ sfn-ecs-python-sam/example-pattern.json | 64 +++ .../statemachine/callback-pattern.asl.json | 87 ++++ .../statemachine/sync-pattern.asl.json | 69 ++++ sfn-ecs-python-sam/template.yaml | 378 ++++++++++++++++++ 5 files changed, 947 insertions(+) create mode 100644 sfn-ecs-python-sam/README.md create mode 100644 sfn-ecs-python-sam/example-pattern.json create mode 100644 sfn-ecs-python-sam/statemachine/callback-pattern.asl.json create mode 100644 sfn-ecs-python-sam/statemachine/sync-pattern.asl.json create mode 100644 sfn-ecs-python-sam/template.yaml diff --git a/sfn-ecs-python-sam/README.md b/sfn-ecs-python-sam/README.md new file mode 100644 index 000000000..edc3b47ee --- /dev/null +++ b/sfn-ecs-python-sam/README.md @@ -0,0 +1,349 @@ +# AWS Step Functions to Amazon ECS with Python + +This pattern demonstrates how to invoke an Amazon ECS task from AWS Step Functions using Python, showcasing both **synchronous (polling)** and **callback** integration patterns. + +Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns + +**Important:** This application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. + +## Requirements + +* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. +* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured +* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed + +## Architecture + +### Pattern 1: Synchronous (Polling) Integration + +``` +┌─────────────┐ ┌──────────────────┐ ┌─────────────┐ +│ Step │ │ ECS Task │ │ CloudWatch │ +│ Functions │─────▶│ (Python) │─────▶│ Logs │ +│ (Sync) │ │ │ │ │ +└─────────────┘ └──────────────────┘ └─────────────┘ + │ │ + │ │ + └───────────────────────┘ + Waits for completion +``` + +**How it works:** +1. Step Functions invokes the ECS task using `arn:aws:states:::ecs:runTask.sync` +2. Step Functions **polls** the task status automatically +3. The workflow **waits** until the ECS task completes +4. Once complete, Step Functions continues to the next state +5. If the task fails, Step Functions can catch the error and retry + +**Use cases:** +- Short to medium-duration tasks (< 1 year) +- When you need the task result before proceeding +- Simple workflows where waiting is acceptable + +### Pattern 2: Callback Integration + +``` +┌─────────────┐ ┌──────────────────┐ ┌─────────────┐ +│ Step │ │ ECS Task │ │ CloudWatch │ +│ Functions │─────▶│ (Python) │─────▶│ Logs │ +│ (Callback) │ │ │ │ │ +└─────────────┘ └──────────────────┘ └─────────────┘ + ▲ │ + │ │ + └───────────────────────┘ + Task sends callback +``` + +**How it works:** +1. Step Functions invokes the ECS task using `arn:aws:states:::ecs:runTask.waitForTaskToken` +2. Step Functions passes a **task token** to the ECS container via environment variable +3. Step Functions **pauses** and waits for a callback +4. The Python application in ECS processes the work +5. When done, the Python app calls `send_task_success()` or `send_task_failure()` with the task token +6. Step Functions receives the callback and continues + +**Use cases:** +- Long-running tasks +- Human approval workflows +- External system integrations +- When you need to decouple task execution from workflow progression + +## Deployment Instructions + +### Step 1: Clone the Repository + +```bash +git clone https://github.com/aws-samples/serverless-patterns +cd serverless-patterns/sfn-ecs-python-sam +``` + +### Step 2: Build and Deploy + +```bash +sam build +sam deploy --guided +``` + +During the prompts: +- **Stack Name**: `sfn-ecs-python-stack` +- **AWS Region**: Your preferred region (e.g., `us-east-1`) +- **Confirm changes before deploy**: Y +- **Allow SAM CLI IAM role creation**: Y +- **Save arguments to samconfig.toml**: Y + +### Step 3: Note the Outputs + +After deployment, note the following outputs: +- `SyncStateMachineArn` - ARN for the synchronous pattern +- `CallbackStateMachineArn` - ARN for the callback pattern +- `ECSClusterName` - Name of the ECS cluster +- `TaskDefinitionArn` - ARN of the task definition + +## How to Test + +### Testing the Synchronous Pattern + +1. **Start the execution:** + +```bash +aws stepfunctions start-execution \ + --state-machine-arn \ + --input '{"message": "Hello from sync pattern", "processingTime": 10}' +``` + +2. **Monitor the execution:** + +```bash +aws stepfunctions describe-execution \ + --execution-arn +``` + +3. **View the output:** + +The execution will wait for the ECS task to complete and return the result: + +```json +{ + "status": "success", + "message": "Processed: Hello from sync pattern", + "processingTime": 10, + "timestamp": "2024-02-05T10:30:00Z" +} +``` + +### Testing the Callback Pattern + +1. **Start the execution:** + +```bash +aws stepfunctions start-execution \ + --state-machine-arn \ + --input '{"message": "Hello from callback pattern", "processingTime": 30}' +``` + +2. **Monitor the execution:** + +```bash +aws stepfunctions describe-execution \ + --execution-arn +``` + +The execution will show status as `RUNNING` while waiting for the callback. + +3. **View ECS task logs:** + +```bash +aws logs tail /ecs/sfn-ecs-python-callback --follow +``` + +4. **The task will automatically send the callback** when processing completes. + +## Step-by-Step Explanation + +### Understanding the Synchronous Pattern + +**Step 1: State Machine Definition** + +The state machine uses the `.sync` integration pattern: + +```json +{ + "Type": "Task", + "Resource": "arn:aws:states:::ecs:runTask.sync", + "Parameters": { + "Cluster": "my-cluster", + "TaskDefinition": "my-task", + "LaunchType": "FARGATE", + "NetworkConfiguration": { + "AwsvpcConfiguration": { + "Subnets": ["subnet-xxx"], + "SecurityGroups": ["sg-xxx"], + "AssignPublicIp": "ENABLED" + } + }, + "Overrides": { + "ContainerOverrides": [{ + "Name": "python-container", + "Environment": [{ + "Name": "MESSAGE", + "Value.$": "$.message" + }] + }] + } + } +} +``` + +**Step 2: ECS Task Execution** + +The Python container starts and processes the input: + +```python +import os +import time +import json + +def main(): + message = os.environ.get('MESSAGE', 'No message') + processing_time = int(os.environ.get('PROCESSING_TIME', '5')) + + print(f"Processing: {message}") + time.sleep(processing_time) + + result = { + "status": "success", + "message": f"Processed: {message}", + "processingTime": processing_time + } + + print(json.dumps(result)) + +if __name__ == "__main__": + main() +``` + +**Step 3: Step Functions Polling** + +- Step Functions automatically polls ECS every few seconds +- Checks if the task is still running +- When the task completes (or fails), Step Functions captures the result +- The workflow continues to the next state + +### Understanding the Callback Pattern + +**Step 1: State Machine Definition** + +The state machine uses the `.waitForTaskToken` integration pattern: + +```json +{ + "Type": "Task", + "Resource": "arn:aws:states:::ecs:runTask.waitForTaskToken", + "Parameters": { + "Cluster": "my-cluster", + "TaskDefinition": "my-task", + "LaunchType": "FARGATE", + "Overrides": { + "ContainerOverrides": [{ + "Name": "python-container", + "Environment": [{ + "Name": "TASK_TOKEN", + "Value.$": "$$.Task.Token" + }, { + "Name": "MESSAGE", + "Value.$": "$.message" + }] + }] + } + } +} +``` + +**Key difference:** The `$$.Task.Token` is passed to the container. + +**Step 2: ECS Task with Callback** + +The Python container receives the task token and sends a callback: + +```python +import os +import boto3 +import json +import time + +def main(): + task_token = os.environ.get('TASK_TOKEN') + message = os.environ.get('MESSAGE', 'No message') + processing_time = int(os.environ.get('PROCESSING_TIME', '5')) + + sfn_client = boto3.client('stepfunctions') + + try: + print(f"Processing: {message}") + time.sleep(processing_time) + + result = { + "status": "success", + "message": f"Processed: {message}", + "processingTime": processing_time + } + + # Send success callback + sfn_client.send_task_success( + taskToken=task_token, + output=json.dumps(result) + ) + + except Exception as e: + # Send failure callback + sfn_client.send_task_failure( + taskToken=task_token, + error='ProcessingError', + cause=str(e) + ) + +if __name__ == "__main__": + main() +``` + +**Step 3: Callback Mechanism** + +- Step Functions pauses execution after starting the ECS task +- The task token acts as a unique identifier for this specific execution +- The Python app calls `send_task_success()` or `send_task_failure()` +- Step Functions receives the callback and resumes execution +- The output from the callback becomes the state output + +## Key Differences Between Patterns + +| Feature | Synchronous (.sync) | Callback (.waitForTaskToken) | +|---------|-------------------|------------------------------| +| **Max Duration** | 1 year | 1 year | +| **Polling** | Automatic by Step Functions | No polling needed | +| **Task Awareness** | Task doesn't know about Step Functions | Task must send callback | +| **Complexity** | Simple | Moderate (requires SDK calls) | +| **Use Case** | Standard batch jobs | Long-running, human approval, external systems | +| **Failure Handling** | Automatic | Manual (task must call send_task_failure) | +| **Cost** | State transitions for polling | Fewer state transitions | + +## Cleanup + +To delete the resources: + +```bash +sam delete +``` + +## Resources + +- [AWS Step Functions](https://aws.amazon.com/step-functions/) +- [Amazon ECS](https://aws.amazon.com/ecs/) +- [Step Functions ECS Integration](https://docs.aws.amazon.com/step-functions/latest/dg/connect-ecs.html) +- [Task Token Pattern](https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-wait-token) + +--- + +Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 \ No newline at end of file diff --git a/sfn-ecs-python-sam/example-pattern.json b/sfn-ecs-python-sam/example-pattern.json new file mode 100644 index 000000000..3f3fcad73 --- /dev/null +++ b/sfn-ecs-python-sam/example-pattern.json @@ -0,0 +1,64 @@ +{ + "title": "AWS Step Functions to Amazon ECS with Python", + "description": "Invoke ECS tasks from Step Functions using synchronous (polling) and callback integration patterns with Python containers", + "language": "Python", + "level": "200", + "framework": "SAM", + "introBox": { + "headline": "How it works", + "text": [ + "This pattern demonstrates two ways to integrate AWS Step Functions with Amazon ECS tasks running Python code:", + "1. Synchronous Pattern (.sync): Step Functions automatically polls the ECS task status and waits for completion", + "2. Callback Pattern (.waitForTaskToken): The ECS task receives a token and explicitly sends a callback when done", + "The pattern includes complete Python code running in ECS Fargate containers, VPC networking, and CloudWatch logging." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/sfn-ecs-python-sam", + "templateURL": "serverless-patterns/sfn-ecs-python-sam", + "projectFolder": "sfn-ecs-python-sam", + "templateFile": "template.yaml" + } + }, + "resources": { + "bullets": [ + { + "text": "Run Amazon ECS or Fargate tasks with Step Functions", + "link": "https://docs.aws.amazon.com/step-functions/latest/dg/connect-ecs.html" + }, + { + "text": "Wait for a Callback with Task Token", + "link": "https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-wait-token" + }, + { + "text": "Amazon ECS Task Definitions", + "link": "https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html" + } + ] + }, + "deploy": { + "text": [ + "sam build", + "sam deploy --guided" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete" + ] + }, + "authors": [ + { + "name": "Mian Tariq", + "image": "", + "bio": "Senior Delivery Consultant", + "linkedin": "" + } + ] +} \ No newline at end of file diff --git a/sfn-ecs-python-sam/statemachine/callback-pattern.asl.json b/sfn-ecs-python-sam/statemachine/callback-pattern.asl.json new file mode 100644 index 000000000..c33c3be11 --- /dev/null +++ b/sfn-ecs-python-sam/statemachine/callback-pattern.asl.json @@ -0,0 +1,87 @@ +{ + "Comment": "Callback ECS Task Execution - Task sends callback when complete", + "StartAt": "Run ECS Task (Callback)", + "States": { + "Run ECS Task (Callback)": { + "Type": "Task", + "Resource": "arn:aws:states:::ecs:runTask.waitForTaskToken", + "Parameters": { + "Cluster": "${ECSCluster}", + "TaskDefinition": "${TaskDefinition}", + "LaunchType": "FARGATE", + "NetworkConfiguration": { + "AwsvpcConfiguration": { + "Subnets": [ + "${Subnet1}", + "${Subnet2}" + ], + "SecurityGroups": [ + "${SecurityGroup}" + ], + "AssignPublicIp": "ENABLED" + } + }, + "Overrides": { + "ContainerOverrides": [ + { + "Name": "python-callback-container", + "Environment": [ + { + "Name": "TASK_TOKEN", + "Value.$": "$$.Task.Token" + }, + { + "Name": "MESSAGE", + "Value.$": "$.message" + }, + { + "Name": "PROCESSING_TIME", + "Value.$": "States.Format('{}', $.processingTime)" + } + ] + } + ] + } + }, + "TimeoutSeconds": 300, + "HeartbeatSeconds": 60, + "Next": "Callback Received Successfully", + "Catch": [ + { + "ErrorEquals": [ + "States.Timeout" + ], + "Next": "Task Timed Out", + "ResultPath": "$.error" + }, + { + "ErrorEquals": [ + "States.ALL" + ], + "Next": "Task Failed", + "ResultPath": "$.error" + } + ] + }, + "Callback Received Successfully": { + "Type": "Pass", + "Parameters": { + "status": "completed", + "pattern": "callback", + "message": "ECS task sent callback successfully", + "result.$": "$" + }, + "End": true + }, + "Task Timed Out": { + "Type": "Fail", + "Error": "TaskTimeout", + "Cause": "The ECS task did not send a callback within the timeout period" + }, + "Task Failed": { + "Type": "Fail", + "Error": "ECSTaskFailed", + "Cause": "The ECS task failed or sent a failure callback" + } + } +} \ No newline at end of file diff --git a/sfn-ecs-python-sam/statemachine/sync-pattern.asl.json b/sfn-ecs-python-sam/statemachine/sync-pattern.asl.json new file mode 100644 index 000000000..55a2961e7 --- /dev/null +++ b/sfn-ecs-python-sam/statemachine/sync-pattern.asl.json @@ -0,0 +1,69 @@ +{ + "Comment": "Synchronous ECS Task Execution - Step Functions polls until task completes", + "StartAt": "Run ECS Task (Sync)", + "States": { + "Run ECS Task (Sync)": { + "Type": "Task", + "Resource": "arn:aws:states:::ecs:runTask.sync", + "Parameters": { + "Cluster": "${ECSCluster}", + "TaskDefinition": "${TaskDefinition}", + "LaunchType": "FARGATE", + "NetworkConfiguration": { + "AwsvpcConfiguration": { + "Subnets": [ + "${Subnet1}", + "${Subnet2}" + ], + "SecurityGroups": [ + "${SecurityGroup}" + ], + "AssignPublicIp": "ENABLED" + } + }, + "Overrides": { + "ContainerOverrides": [ + { + "Name": "python-sync-container", + "Environment": [ + { + "Name": "MESSAGE", + "Value.$": "$.message" + }, + { + "Name": "PROCESSING_TIME", + "Value.$": "States.Format('{}', $.processingTime)" + } + ] + } + ] + } + }, + "Next": "Task Completed Successfully", + "Catch": [ + { + "ErrorEquals": [ + "States.ALL" + ], + "Next": "Task Failed", + "ResultPath": "$.error" + } + ] + }, + "Task Completed Successfully": { + "Type": "Pass", + "Parameters": { + "status": "completed", + "pattern": "synchronous", + "message": "ECS task completed successfully", + "input.$": "$" + }, + "End": true + }, + "Task Failed": { + "Type": "Fail", + "Error": "ECSTaskFailed", + "Cause": "The ECS task failed to complete successfully" + } + } +} \ No newline at end of file diff --git a/sfn-ecs-python-sam/template.yaml b/sfn-ecs-python-sam/template.yaml new file mode 100644 index 000000000..4c6662767 --- /dev/null +++ b/sfn-ecs-python-sam/template.yaml @@ -0,0 +1,378 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: > + Step Functions to ECS with Python - Demonstrates sync and callback patterns + +Parameters: + VpcCIDR: + Type: String + Default: 10.0.0.0/16 + Description: CIDR block for VPC + +Resources: + # VPC and Networking + VPC: + Type: AWS::EC2::VPC + Properties: + CidrBlock: !Ref VpcCIDR + EnableDnsHostnames: true + EnableDnsSupport: true + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-vpc + + PublicSubnet1: + Type: AWS::EC2::Subnet + Properties: + VpcId: !Ref VPC + CidrBlock: 10.0.1.0/24 + AvailabilityZone: !Select [0, !GetAZs ''] + MapPublicIpOnLaunch: true + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-public-subnet-1 + + PublicSubnet2: + Type: AWS::EC2::Subnet + Properties: + VpcId: !Ref VPC + CidrBlock: 10.0.2.0/24 + AvailabilityZone: !Select [1, !GetAZs ''] + MapPublicIpOnLaunch: true + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-public-subnet-2 + + InternetGateway: + Type: AWS::EC2::InternetGateway + Properties: + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-igw + + AttachGateway: + Type: AWS::EC2::VPCGatewayAttachment + Properties: + VpcId: !Ref VPC + InternetGatewayId: !Ref InternetGateway + + PublicRouteTable: + Type: AWS::EC2::RouteTable + Properties: + VpcId: !Ref VPC + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-public-rt + + PublicRoute: + Type: AWS::EC2::Route + DependsOn: AttachGateway + Properties: + RouteTableId: !Ref PublicRouteTable + DestinationCidrBlock: 0.0.0.0/0 + GatewayId: !Ref InternetGateway + + SubnetRouteTableAssociation1: + Type: AWS::EC2::SubnetRouteTableAssociation + Properties: + SubnetId: !Ref PublicSubnet1 + RouteTableId: !Ref PublicRouteTable + + SubnetRouteTableAssociation2: + Type: AWS::EC2::SubnetRouteTableAssociation + Properties: + SubnetId: !Ref PublicSubnet2 + RouteTableId: !Ref PublicRouteTable + + # Security Group + ECSSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for ECS tasks + VpcId: !Ref VPC + SecurityGroupEgress: + - IpProtocol: -1 + CidrIp: 0.0.0.0/0 + Tags: + - Key: Name + Value: !Sub ${AWS::StackName}-ecs-sg + + # ECS Cluster + ECSCluster: + Type: AWS::ECS::Cluster + Properties: + ClusterName: !Sub ${AWS::StackName}-cluster + ClusterSettings: + - Name: containerInsights + Value: enabled + + # CloudWatch Log Group + ECSLogGroup: + Type: AWS::Logs::LogGroup + Properties: + LogGroupName: !Sub /ecs/${AWS::StackName} + RetentionInDays: 7 + + # ECS Task Execution Role + ECSTaskExecutionRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: ecs-tasks.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy + + # ECS Task Role (for callback pattern) + ECSTaskRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: ecs-tasks.amazonaws.com + Action: sts:AssumeRole + Policies: + - PolicyName: StepFunctionsCallback + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - states:SendTaskSuccess + - states:SendTaskFailure + - states:SendTaskHeartbeat + Resource: '*' + + # ECS Task Definition for Sync Pattern + SyncTaskDefinition: + Type: AWS::ECS::TaskDefinition + Properties: + Family: !Sub ${AWS::StackName}-sync-task + NetworkMode: awsvpc + RequiresCompatibilities: + - FARGATE + Cpu: '256' + Memory: '512' + ExecutionRoleArn: !GetAtt ECSTaskExecutionRole.Arn + ContainerDefinitions: + - Name: python-sync-container + Image: public.ecr.aws/docker/library/python:3.11-slim + Essential: true + Command: + - /bin/sh + - -c + - | + pip install --quiet boto3 && python3 << 'EOF' + import os + import time + import json + from datetime import datetime + + def main(): + message = os.environ.get('MESSAGE', 'No message provided') + processing_time = int(os.environ.get('PROCESSING_TIME', '5')) + + print(f"[SYNC] Starting processing: {message}") + print(f"[SYNC] Will process for {processing_time} seconds") + + # Simulate processing + time.sleep(processing_time) + + result = { + "status": "success", + "message": f"Processed: {message}", + "processingTime": processing_time, + "timestamp": datetime.utcnow().isoformat() + "Z" + } + + print(f"[SYNC] Completed: {json.dumps(result)}") + return result + + if __name__ == "__main__": + main() + EOF + LogConfiguration: + LogDriver: awslogs + Options: + awslogs-group: !Ref ECSLogGroup + awslogs-region: !Ref AWS::Region + awslogs-stream-prefix: sync + + # ECS Task Definition for Callback Pattern + CallbackTaskDefinition: + Type: AWS::ECS::TaskDefinition + Properties: + Family: !Sub ${AWS::StackName}-callback-task + NetworkMode: awsvpc + RequiresCompatibilities: + - FARGATE + Cpu: '256' + Memory: '512' + ExecutionRoleArn: !GetAtt ECSTaskExecutionRole.Arn + TaskRoleArn: !GetAtt ECSTaskRole.Arn + ContainerDefinitions: + - Name: python-callback-container + Image: public.ecr.aws/docker/library/python:3.11-slim + Essential: true + Command: + - /bin/sh + - -c + - | + pip install --quiet boto3 && python3 << 'EOF' + import os + import boto3 + import json + import time + from datetime import datetime + + def main(): + task_token = os.environ.get('TASK_TOKEN') + message = os.environ.get('MESSAGE', 'No message provided') + processing_time = int(os.environ.get('PROCESSING_TIME', '5')) + + if not task_token: + print("[CALLBACK] ERROR: No task token provided!") + return + + sfn_client = boto3.client('stepfunctions') + + try: + print(f"[CALLBACK] Starting processing: {message}") + print(f"[CALLBACK] Task token: {task_token[:20]}...") + print(f"[CALLBACK] Will process for {processing_time} seconds") + + # Simulate processing + time.sleep(processing_time) + + result = { + "status": "success", + "message": f"Processed: {message}", + "processingTime": processing_time, + "timestamp": datetime.utcnow().isoformat() + "Z" + } + + print(f"[CALLBACK] Sending success callback") + sfn_client.send_task_success( + taskToken=task_token, + output=json.dumps(result) + ) + print(f"[CALLBACK] Success callback sent!") + + except Exception as e: + print(f"[CALLBACK] ERROR: {str(e)}") + try: + sfn_client.send_task_failure( + taskToken=task_token, + error='ProcessingError', + cause=str(e) + ) + print(f"[CALLBACK] Failure callback sent") + except Exception as callback_error: + print(f"[CALLBACK] Failed to send callback: {str(callback_error)}") + + if __name__ == "__main__": + main() + EOF + LogConfiguration: + LogDriver: awslogs + Options: + awslogs-group: !Ref ECSLogGroup + awslogs-region: !Ref AWS::Region + awslogs-stream-prefix: callback + + # Step Functions Role + StepFunctionsRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: states.amazonaws.com + Action: sts:AssumeRole + Policies: + - PolicyName: ECSTaskExecution + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - ecs:RunTask + - ecs:StopTask + - ecs:DescribeTasks + Resource: '*' + - Effect: Allow + Action: + - iam:PassRole + Resource: + - !GetAtt ECSTaskExecutionRole.Arn + - !GetAtt ECSTaskRole.Arn + - Effect: Allow + Action: + - events:PutTargets + - events:PutRule + - events:DescribeRule + Resource: '*' + + # Synchronous Pattern State Machine + SyncStateMachine: + Type: AWS::Serverless::StateMachine + Properties: + Name: !Sub ${AWS::StackName}-sync-pattern + Type: STANDARD + Role: !GetAtt StepFunctionsRole.Arn + DefinitionUri: statemachine/sync-pattern.asl.json + DefinitionSubstitutions: + ECSCluster: !Ref ECSCluster + TaskDefinition: !Ref SyncTaskDefinition + Subnet1: !Ref PublicSubnet1 + Subnet2: !Ref PublicSubnet2 + SecurityGroup: !Ref ECSSecurityGroup + + # Callback Pattern State Machine + CallbackStateMachine: + Type: AWS::Serverless::StateMachine + Properties: + Name: !Sub ${AWS::StackName}-callback-pattern + Type: STANDARD + Role: !GetAtt StepFunctionsRole.Arn + DefinitionUri: statemachine/callback-pattern.asl.json + DefinitionSubstitutions: + ECSCluster: !Ref ECSCluster + TaskDefinition: !Ref CallbackTaskDefinition + Subnet1: !Ref PublicSubnet1 + Subnet2: !Ref PublicSubnet2 + SecurityGroup: !Ref ECSSecurityGroup + +Outputs: + SyncStateMachineArn: + Description: ARN of the Synchronous Pattern State Machine + Value: !Ref SyncStateMachine + + CallbackStateMachineArn: + Description: ARN of the Callback Pattern State Machine + Value: !Ref CallbackStateMachine + + ECSClusterName: + Description: Name of the ECS Cluster + Value: !Ref ECSCluster + + SyncTaskDefinitionArn: + Description: ARN of the Sync Task Definition + Value: !Ref SyncTaskDefinition + + CallbackTaskDefinitionArn: + Description: ARN of the Callback Task Definition + Value: !Ref CallbackTaskDefinition + + LogGroupName: + Description: CloudWatch Log Group for ECS tasks + Value: !Ref ECSLogGroup \ No newline at end of file From ce2829ff3059c34e77efcf7c213c3596d538fea0 Mon Sep 17 00:00:00 2001 From: 4D54 <78043109+4D54@users.noreply.github.com> Date: Thu, 5 Feb 2026 20:14:41 +0100 Subject: [PATCH 2/2] Add security note to README - Document permissive IAM and security group configurations - Explain why they exist (demo/learning purposes) - Provide production security recommendations - Recommend deployment in non-production environments --- sfn-ecs-python-sam/README.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/sfn-ecs-python-sam/README.md b/sfn-ecs-python-sam/README.md index edc3b47ee..01b84f6d9 100644 --- a/sfn-ecs-python-sam/README.md +++ b/sfn-ecs-python-sam/README.md @@ -6,6 +6,22 @@ Learn more about this pattern at Serverless Land Patterns: https://serverlesslan **Important:** This application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. +## Security Note + +This pattern is designed for learning and demonstration purposes. The IAM roles and security group use permissive configurations to simplify deployment and focus on the integration patterns: + +- **Security Group**: Allows all outbound traffic (required for pulling Docker images and calling AWS APIs) +- **IAM Roles**: Use wildcard (`*`) resources for ECS task management and Step Functions callbacks + +**For production use**, you should: +- Restrict security group egress to specific AWS service endpoints using VPC endpoints +- Scope IAM policies to specific resources (task definitions, state machines) +- Implement least privilege access based on your security requirements +- Consider using AWS PrivateLink for service-to-service communication +- Enable VPC Flow Logs for network traffic monitoring + +Deploy this pattern in a non-production AWS account or isolated environment for testing. + ## Requirements * [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.