diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 30a6a6e3..7537e8c7 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -36,17 +36,21 @@ jobs: - name: Setup Node.js uses: actions/setup-node@v4 with: - node-version: 18 + node-version: 20 + - name: Install LocalStack run: pip install localstack awscli-local[ver1] virtualenv + - name: Setup config run: | echo "Configuring git for codecommit sample" git config --global user.email "localstack.sample@localstack.cloud" git config --global user.name "Localstack Pro-Samples" + - name: Pull the latest docker image run: docker pull localstack/localstack-pro - - name: Execute a simple test + + - name: Execute tests timeout-minutes: 10 run: | cd ${{ matrix.directory }} diff --git a/README.md b/README.md index d3d53689..2ec90e8e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # LocalStack Pro Samples -This repository contains sample projects that can be deployed on your local machine using [LocalStack Pro](https://localstack.cloud/). +This repository contains sample projects that can be deployed on your local machine using [LocalStack](https://localstack.cloud/). Each example in the repository is prefixed with the name of the AWS service being used. For example, the `elb-load-balancing` directory contains examples that demonstrate how to use the Elastic Load Balancing service with LocalStack. Please refer to the sub directories for more details and instructions on how to start the samples. @@ -14,7 +14,19 @@ Each example in the repository is prefixed with the name of the AWS service bein ## Configuration -Some of the samples require LocalStack Pro features. Please make sure to properly configure the `LOCALSTACK_AUTH_TOKEN` environment variable. You can find your Auth Token on the [LocalStack Web Application](https://app.localstack.cloud/workspace/auth-token) and you can refer to our [Auth Token documentation](https://docs.localstack.cloud/getting-started/auth-token/) for more details. +All samples require a valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. Set it before running any sample: + +```bash +export LOCALSTACK_AUTH_TOKEN= +``` + +Alternatively, use the LocalStack CLI to persist the token: + +```bash +localstack auth set-token +``` + +You can find your Auth Token on the [LocalStack Web Application](https://app.localstack.cloud/workspace/auth-token). ## Outline @@ -22,6 +34,7 @@ Some of the samples require LocalStack Pro features. Please make sure to properl | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | [Serverless Websockets](serverless-websockets) | API Gateway V2 websocket APIs deployed via the Serverless framework | | [RDS Database Queries](rds-db-queries) | Running queries locally against an RDS database | +| [RDS Failover Test](rds-failover-test) | Running a failover test against an RDS global cluster | | [Neptune Graph Database](neptune-graph-db) | Running queries locally against a Neptune Graph database | | [Lambda Event Filtering](lambda-event-filtering) | Lambda event source filtering with DynamoDB and SQS | | [Glacier & S3 select queries](glacier-s3-select) | Using Glacier API and running S3 Select queries locally | @@ -42,7 +55,9 @@ Some of the samples require LocalStack Pro features. Please make sure to properl | [ECS ECR Container application](ecs-ecr-container-app) | Pushing Docker images to ECR and running them locally on ECS | | [Athena queries over S3](athena-s3-queries) | Running Athena queries over S3 files locally | | [Terraform resources](terraform-resources) | Deploying various AWS resources via Terraform | -| [Lambda Function URLs](lambda-function-urls) | Invoking Lambda functions via HTTP(s) URLs | +| [CDK for Terraform resources](cdk-for-terraform) | Deploying AWS resources via CDK for Terraform | +| [Lambda Function URLs (JavaScript)](lambda-function-urls-javascript) | Invoking Lambda functions via HTTP(S) URLs using JavaScript | +| [Lambda Function URLs (Python)](lambda-function-urls-python) | Invoking Lambda functions via HTTP(S) URLs using Python | | [Sagemaker inference](sagemaker-inference) | Creating & invoking a Sagemaker endpoint locally with MNIST dataset | | [MSK with Glue Schema Registry](glue-msk-schema-registry) | Use of MSK, Glue Schema Registry, Glue ETL, and RDS | | [AppSync GraphQL](appsync-graphql-api) | Deploying a GraphQL API using AppSync | @@ -59,6 +74,13 @@ Some of the samples require LocalStack Pro features. Please make sure to properl | [ELB Load Balancing](elb-load-balancing) | Using ELBv2 Application Load Balancers locally, deployed via the Serverless framework | | [Reproducible ML](reproducible-ml) | Train, save and evaluate a scikit-learn machine learning model using AWS Lambda and S3 | | [Lambda PHP/Bref CDK App](lambda-php-bref-cdk-app) | Running PHP/Bref Lambda handler locally, deployed via AWS CDK | +| [Step Functions with Lambda](stepfunctions-lambda) | Orchestrating Lambda functions using AWS Step Functions | +| [Multi-Account S3 Access](multi-account-multi-region-s3-access) | Accessing S3 resources across different AWS accounts and regions | +| [Route53 DNS Failover](route53-dns-failover) | Route53 DNS failover based on health checks | +| [EMR Serverless Sample](emr-serverless-sample) | Running EMR Serverless jobs locally | +| [EMR Serverless Spark](emr-serverless-spark) | Running Java Spark jobs on EMR Serverless | +| [EMR Serverless Python Dependencies](emr-serverless-python-dependencies) | Adding Python dependencies to EMR Serverless PySpark jobs | +| [Testcontainers Java Sample](testcontainers-java-sample) | Using LocalStack Testcontainers with RDS in Java | ## Checking out a single sample @@ -104,6 +126,7 @@ run: ## Run the actual sample steps/commands. This assumes LocalStack is ./run.sh start: ## Start LocalStack in detached mode + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Get your auth token at https://localstack.cloud/pricing."; exit 1) localstack start -d stop: ## Stop the Running LocalStack container @@ -117,9 +140,9 @@ ready: ## Make sure the LocalStack container is up logs: ## Save the logs in a separate file, since the LS container will only contain the logs of the last sample run. @localstack logs > logs.txt -test-ci: ## Execute the necessary targets in the correct order for an automatic execution. +test-ci: ## Execute the necessary targets in the correct order for an automatic execution. make start install ready run; return_code=`echo $$?`;\ make logs; make stop; exit $$return_code; .PHONY: usage install run start stop ready logs test-ci -``` \ No newline at end of file +``` diff --git a/apigw-custom-domain/Makefile b/apigw-custom-domain/Makefile index a565f73b..de2b3f97 100644 --- a/apigw-custom-domain/Makefile +++ b/apigw-custom-domain/Makefile @@ -55,7 +55,8 @@ test: ## Run tests make deploy run target=ci start: ## Start LocalStack - ACTIVATE_PRO=1 localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d stop: ## Stop LocalStack @echo diff --git a/apigw-custom-domain/README.md b/apigw-custom-domain/README.md index cc0f813e..8826e7af 100644 --- a/apigw-custom-domain/README.md +++ b/apigw-custom-domain/README.md @@ -2,7 +2,6 @@ | Key | Value | | ------------ | --------------------------------- | -| Environment | LocalStack | | Services | API Gateway, Lambda, Route53, ACM | | Integrations | Serverless Framework | | Categories | Serverless; REST API | @@ -15,9 +14,13 @@ Under the hood, the Serverless framework uses the [`serverless-localstack`](http ## Prerequisites -* [Node.js 18.x](https://nodejs.org/en/download/package-manager) with `npm` -* [Serverless Framework](https://www.serverless.com/framework/docs/getting-started) 3.x -* `openssl` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Node.js 18.x](https://nodejs.org/en/download/package-manager) with `npm` +- [Serverless Framework](https://www.serverless.com/framework/docs/getting-started) 3.x +- `openssl` ## Check prerequisites @@ -34,6 +37,7 @@ make install ## Start LocalStack ```bash +export LOCALSTACK_AUTH_TOKEN= make start ``` diff --git a/appsync-graphql-api/Makefile b/appsync-graphql-api/Makefile index ae1797c1..764cc898 100644 --- a/appsync-graphql-api/Makefile +++ b/appsync-graphql-api/Makefile @@ -7,6 +7,15 @@ SHELL := /bin/bash usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @test -e node_modules || npm install @which serverless || npm install -g serverless @@ -17,7 +26,8 @@ install: ## Install dependencies run: ## Deploy the app locally and run an AppSync GraphQL test invocation ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/appsync-graphql-api/README.md b/appsync-graphql-api/README.md index 4675a592..1b240d35 100644 --- a/appsync-graphql-api/README.md +++ b/appsync-graphql-api/README.md @@ -1,42 +1,58 @@ -# LocalStack Demo: AppSync GraphQL APIs for DynamoDB and RDS Aurora Postgres +# AppSync GraphQL API -Simple demo application illustrating how to proxy data from different resources (DynamoDB tables, RDS databases) via AppSync GraphQL using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | AppSync, DynamoDB, RDS | +| Integrations | AWS SDK, AWS CLI | +| Categories | GraphQL; Serverless | + +## Introduction + +A demo application illustrating how to proxy data from different resources (DynamoDB tables, RDS Aurora Postgres databases) via AppSync GraphQL using LocalStack. The sample runs mutation and query operations for two data sources and demonstrates real-time notifications via WebSocket subscriptions. ## Prerequisites -* LocalStack -* Docker -* Python 3.6+ -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Deploy the app locally and run the GraphQL test invocations: -``` +```bash make run ``` -The demo will run different GraphQL queries, for two different datasources (DynamoDB / RDS Aurora): +The script: -1. a mutation query which inserts a new item into DynamoDB / RDS Aurora -2. a query which scans and returns the items from DynamoDB / RDS Aurora +- Deploys AppSync GraphQL API with DynamoDB and RDS Aurora Postgres resolvers. +- Runs mutation queries to insert items into both data sources. +- Runs query operations to scan and return items from DynamoDB and RDS Aurora. +- Connects a WebSocket client to verify real-time subscription notifications. You should see a success output in the terminal: -``` + +```bash {"data":{"addPostDDB":{"id":{"S":"id123"}}}} {"data":{"getPostsDDB":[{"id":{"S":"id123"}}]}} ... @@ -44,8 +60,9 @@ You should see a success output in the terminal: {"data":{"getPostsRDS":[{"id":{"S":"id123"}}]}} ``` -... and the item should have been added to your local DynamoDB table (as well as your RDS database): -``` +The item should also have been added to your local DynamoDB table: + +```bash $ awslocal dynamodb scan --table-name table1 { "Items": [ @@ -61,9 +78,9 @@ $ awslocal dynamodb scan --table-name table1 } ``` -Finally, you should also see a message printed from the WebSocket client subscribed to notifications from the API: -``` -... +Finally, you should see a notification from the WebSocket client: + +```bash Starting a WebSocket client to subscribe to GraphQL mutation operations. Connecting to WebSocket URL ws://localhost:4510/graphql/... ... diff --git a/athena-s3-queries/Makefile b/athena-s3-queries/Makefile index ac209a9e..d9a23343 100644 --- a/athena-s3-queries/Makefile +++ b/athena-s3-queries/Makefile @@ -6,6 +6,13 @@ export AWS_DEFAULT_REGION=us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies # @test -e node_modules || npm install @which localstack || pip install localstack @@ -16,7 +23,8 @@ run: ## Set up database connections and run Athena queries locally echo "Running Athena queries"; \ ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/athena-s3-queries/README.md b/athena-s3-queries/README.md index 7c8e2e30..84693afe 100644 --- a/athena-s3-queries/README.md +++ b/athena-s3-queries/README.md @@ -1,43 +1,57 @@ -# LocalStack Demo: Athena Queries over S3 Files +# Athena Queries over S3 Files -Simple demo application illustrating how to run Athena queries over S3 files locally, using LocalStack. +| Key | Value | +| ------------ | ------------------------------ | +| Services | Athena, S3 | +| Integrations | AWS CLI | +| Categories | Analytics; Serverless | + +## Introduction + +A demo application illustrating how to run Athena queries over S3 files locally using LocalStack. The sample uploads CSV test data to S3, creates Athena table metadata, and runs SQL queries to aggregate results — all without connecting to AWS. ## Prerequisites -* LocalStack -* Docker -* Node.js / `npm` -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Node.js](https://nodejs.org/en/download/) with `npm` -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Start the app locally and run the Athena test queries: -``` +```bash make run ``` -The demo script performs the following actions: +The script: + +- Creates an S3 bucket and uploads CSV test data (person details) to the bucket. +- Runs queries to create Athena database and table metadata. +- Executes a SELECT query to count users by gender. +- Downloads and displays the query results from the S3 results bucket. -1. Create an S3 bucket and upload test data (CSV files with person details) to the bucket -2. Run queries to create the table metadata in Athena -3. Running a simple query over the test file - querying the number of users by gender (male/female) -4. Downloading the query results from the S3 results bucket +You should see output similar to: -You should see something similar to the following log output in the terminal: ``` $ ./run.sh Uploading test data to S3... @@ -46,8 +60,6 @@ upload: data/data.csv to s3://athena-test/data/data.csv Running queries to create database and table definitions... NOTE: This can take a very long time (several minutes) as the system is initializing Waiting for completion status of query cda0572a: RUNNING -Waiting for completion status of query cda0572a: RUNNING -Waiting for completion status of query cda0572a: RUNNING ... Waiting for completion status of query cda0572a: SUCCEEDED Starting SELECT query over data in S3. Query ID: 8a19e3a3 diff --git a/cdk-for-terraform/Makefile b/cdk-for-terraform/Makefile new file mode 100644 index 00000000..487f6aa2 --- /dev/null +++ b/cdk-for-terraform/Makefile @@ -0,0 +1,46 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v terraform > /dev/null 2>&1 || { echo "Terraform is not installed. Please install Terraform and try again."; exit 1; } + @command -v cdktf > /dev/null 2>&1 || { echo "cdktf is not installed. Please install it and try again."; exit 1; } + @command -v pipenv > /dev/null 2>&1 || { echo "pipenv is not installed. Please run: pip install pipenv"; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install dependencies + @cd python && pipenv install + @cd python && cdktf get + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Deploy CDK for Terraform stack + @cd python && cdktf deploy --auto-approve + +run: ## Run the deployed stack (alias for deploy) + make deploy + +test-ci: ## Run CI tests + make check start install ready deploy; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/cdk-for-terraform/README.md b/cdk-for-terraform/README.md new file mode 100644 index 00000000..4aa497cb --- /dev/null +++ b/cdk-for-terraform/README.md @@ -0,0 +1,54 @@ +# CDK for Terraform Resources + +| Key | Value | +| ------------ | -------------------------------------- | +| Services | SNS, DynamoDB, VPC | +| Integrations | CDK for Terraform, Terraform, Pipenv | +| Categories | IaC; Serverless | + +## Introduction + +A demo application illustrating deployment of AWS resources (SNS, DynamoDB, VPC) using [CDK for Terraform](https://developer.hashicorp.com/terraform/cdktf) with LocalStack. CDK for Terraform lets you define infrastructure using familiar programming languages and generates Terraform configuration from it. + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [Terraform](https://developer.hashicorp.com/terraform/downloads) +- [`cdktf`](https://developer.hashicorp.com/terraform/tutorials/cdktf/cdktf-install) +- [`pipenv`](https://pipenv.pypa.io/en/latest/) + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` + +## Deploy the Application + +```bash +make deploy +``` + +The script: + +- Synthesizes CDK for Terraform constructs into Terraform JSON configuration. +- Deploys an SNS topic, DynamoDB table, and VPC to LocalStack using Terraform. + +## License + +This code is available under the Apache 2.0 license. diff --git a/cdk-resources/Makefile b/cdk-resources/Makefile new file mode 100644 index 00000000..02fcc3d6 --- /dev/null +++ b/cdk-resources/Makefile @@ -0,0 +1,46 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v cdklocal > /dev/null 2>&1 || { echo "cdklocal is not installed. Please run: npm install -g aws-cdk-local"; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install dependencies + @npm install + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Deploy CDK resources to LocalStack + @cdklocal bootstrap + @cdklocal deploy --all --require-approval never + +run: ## Run the deployed resources (alias for deploy) + make deploy + +test-ci: ## Run CI tests + make check start install ready deploy; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/cdk-resources/README.md b/cdk-resources/README.md index 7f868f96..14d0e328 100644 --- a/cdk-resources/README.md +++ b/cdk-resources/README.md @@ -1,38 +1,50 @@ -# LocalStack Demo: Deploying Resources via CDK +# CDK Resources -Simple demo application illustrating deployment of AWS CDK resources locally using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda, SQS, SNS, AppSync | +| Integrations | AWS CDK | +| Categories | IaC; Serverless | + +## Introduction + +A demo application illustrating deployment of AWS resources locally using [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/home.html) with LocalStack. The [`cdklocal`](https://github.com/localstack/aws-cdk-local) wrapper is used to redirect CDK deployments to the local LocalStack endpoint. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`cdklocal`](https://github.com/localstack/aws-cdk-local) -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`cdklocal`](https://github.com/localstack/aws-cdk-local) — install with `npm install -g aws-cdk-local` +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Node.js](https://nodejs.org/en/download/) with `npm` -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Deploy the Application -Bootstrap and deploy the CDK app locally: -``` -cdklocal bootstrap -cdklocal deploy +```bash +make deploy ``` -More details following soon. +The script bootstraps and deploys the CDK app to LocalStack using `cdklocal`. ## License diff --git a/chalice-rest-api/Makefile b/chalice-rest-api/Makefile new file mode 100644 index 00000000..e9860c5b --- /dev/null +++ b/chalice-rest-api/Makefile @@ -0,0 +1,44 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v chalice-local > /dev/null 2>&1 || { echo "chalice-local is not installed. Please run: pip install chalice-local"; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install dependencies + @pip install -r requirements-dev.txt + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Deploy the Chalice REST API to LocalStack + @chalice-local deploy + +run: ## Start the Chalice local development server + @chalice-local local + +test-ci: ## Run CI tests + make check start install ready deploy; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/chalice-rest-api/README.md b/chalice-rest-api/README.md index 6b8ed278..4ca304a6 100644 --- a/chalice-rest-api/README.md +++ b/chalice-rest-api/README.md @@ -1,41 +1,56 @@ -# LocalStack Demo: Chalice REST API +# Chalice REST API -Simple demo application illustrating AWS Chalice integration in LocalStack. The AWS Chalice integration features a REST API which can be tested locally and put to production using the [LocalStack's AWS Chalice client](https://github.com/localstack/chalice-local). +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda, API Gateway | +| Integrations | Chalice | +| Categories | REST API; Serverless | -## Prerequisites +## Introduction + +A demo application illustrating the [AWS Chalice](https://github.com/aws/chalice) framework integration with LocalStack. The [`chalice-local`](https://github.com/localstack/chalice-local) client deploys and serves the REST API locally, enabling full local development and testing without connecting to AWS. -- LocalStack -- Docker -- `chalice-local` +## Prerequisites -## Installing +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`chalice-local`](https://github.com/localstack/chalice-local) — install with `pip install chalice-local` +- [Python 3](https://www.python.org/downloads/) -To install the dependencies: +## Check prerequisites -```sh -pip3 install -r requirements-dev.txt +```bash +make check ``` -## Running +## Installation -Make sure that LocalStack is started: +```bash +make install +``` + +## Start LocalStack -```sh -localstack start -d +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -Start the local-server via: +## Deploy the Application -```sh -chalice-local local +```bash +make deploy ``` -You will see the following logs on the terminal: +## Run the application -```sh -Serving on http://127.0.0.1:8000 +```bash +make run ``` +The local development server starts and serves the API at `http://127.0.0.1:8000`. + ## License This code is available under the Apache 2.0 license. diff --git a/cloudwatch-metrics-aws/Makefile b/cloudwatch-metrics-aws/Makefile new file mode 100644 index 00000000..47444adc --- /dev/null +++ b/cloudwatch-metrics-aws/Makefile @@ -0,0 +1,41 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @command -v jq > /dev/null 2>&1 || { echo "jq is not installed. Please install jq and try again."; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install dependencies + @which awslocal || pip install awscli-local + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +run: ## Run the CloudWatch metrics alarm demo + ./run.sh + +test-ci: ## Run CI tests + make check start install ready run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs run test-ci diff --git a/cloudwatch-metrics-aws/README.md b/cloudwatch-metrics-aws/README.md index d287ddf1..a87d090d 100644 --- a/cloudwatch-metrics-aws/README.md +++ b/cloudwatch-metrics-aws/README.md @@ -1,32 +1,78 @@ -# Creating Cloudwatch metric alarms +# CloudWatch Metrics Alarm -This is a simple example for creating cloudwatch metric alarm. -The example creates an alarm based on the metrics of a failing lambda. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | CloudWatch, Lambda, SNS, SES | +| Integrations | AWS CLI | +| Categories | Monitoring; Serverless | -In other words: once the lambda fails, the alarm will be triggered - in our example we use SNS notification via email. +## Introduction +A demo application illustrating how to create a CloudWatch metric alarm that triggers when a Lambda function fails using LocalStack. The alarm sends an SNS notification via email when the Lambda error count exceeds the threshold. + +To receive email notifications locally, you need a mock SMTP server such as [smtp4dev](https://github.com/rnwood/smtp4dev) or [Papercut SMTP](https://github.com/ChangemakerStudios/Papercut-SMTP). Configure the following environment variables before starting LocalStack: + +- `SMTP_HOST`: hostname and port of your mock SMTP server (e.g., `host.docker.internal:2525`) +- `SMTP_USER`: SMTP username (optional) +- `SMTP_PASS`: SMTP password (optional) ## Prerequisites -For this example you will need: -* [LocalStack Pro](https://localstack.cloud), to send emails via SMTP and SES. -* The [awslocal](https://docs.localstack.cloud/integrations/aws-cli/#localstack-aws-cli-awslocal) command line utility -* A mock SMTP server like [smtp4dev](https://github.com/rnwood/smtp4dev) or [Papercut SMTP](https://github.com/ChangemakerStudios/Papercut-SMTP) to receive the email notifications locally. +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [`jq`](https://stedolan.github.io/jq/) + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack + +To receive email notifications locally, start a mock SMTP server such as [smtp4dev](https://github.com/rnwood/smtp4dev) first: + +```bash +docker run --rm -it -p 3000:80 -p 2525:25 rnwood/smtp4dev +``` + +Navigating to `http://localhost:3000` will open a UI to view email notifications. + +Then set `SMTP_HOST` to the SMTP server address and start LocalStack: + +```bash +export LOCALSTACK_AUTH_TOKEN= +export SMTP_HOST=host.docker.internal:2525 +make start +``` -To connect LocalStack with the SMTP server, you need to [configure the following SMTP environment variables](https://docs.localstack.cloud/aws/ses/#pro) when starting LocalStack: - * `SMTP_HOST` this should contain the hostname and the port of your mock SMTP server - * `SMTP_USER` optional, if there is user to connect - * `SMTP_PASS` optional +> [!NOTE] +> If you start LocalStack in Docker mode, it should be possible to use `host.docker.internal`. For most operating systems this should resolve the address correctly, e.g.: `SMTP_HOST=host.docker.internal:2525`. -For example, when using smtp4dev, simply run: +Alternatively, you can use your real SMTP server by setting `SMTP_HOST`, `SMTP_USER`, and `SMTP_PASS`. - docker run --rm -it -p 3000:80 -p 2525:25 rnwood/smtp4dev +## Run the application -Navigating to `http://localhost:3000` will open a UI to access the email notifications. +```bash +make run +``` -Set the environment variable `SMTP_HOST=:2525` to start LocalStack, where `host-name` is the name or IP of the host where the SMTP server can be reached. +The script: -If you start LocalStack in docker mode, it should be possible to use `host.docker.internal`. For most operating systems this should resolve the address correctly, e.g.: `SMTP_HOST=host.docker.internal:2525`. +- Creates a failing Lambda function. +- Creates an SNS topic and subscribes an email address. +- Creates a CloudWatch alarm based on Lambda error metrics. +- Invokes the Lambda function to trigger the alarm. +- Waits for the alarm state to change and confirms the notification. +## License -Alternatively, you can use your real smtp server. Please refer to your provider to set the proper values for `SMTP_HOST`, `SMTP_USER` and `SMTP_PASS` +This code is available under the Apache 2.0 license. diff --git a/codecommit-git-repo/Makefile b/codecommit-git-repo/Makefile index 275dd1e9..b4122bcb 100644 --- a/codecommit-git-repo/Makefile +++ b/codecommit-git-repo/Makefile @@ -6,6 +6,14 @@ export REPO_FOLDER = /tmp/test.codecommit.repo1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v git > /dev/null 2>&1 || { echo "Git is not installed. Please install Git and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -27,7 +35,8 @@ run: ## Deploy and run the sample locally start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/codecommit-git-repo/README.md b/codecommit-git-repo/README.md index 1e6d793f..2d265954 100644 --- a/codecommit-git-repo/README.md +++ b/codecommit-git-repo/README.md @@ -1,33 +1,57 @@ -# LocalStack Demo: Manage Files in a CodeCommit Git Repository +# CodeCommit Git Repository -Simple demo application illustrating the use of the AWS CodeCommit API in LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | CodeCommit | +| Integrations | AWS CLI, Git | +| Categories | DevOps; Source Control | + +## Introduction + +A demo application illustrating the use of the AWS CodeCommit API with LocalStack. The sample creates a Git repository via CodeCommit, commits and pushes files to it, and clones it in a fresh directory — all locally without connecting to AWS. ## Prerequisites -* LocalStack -* Docker -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Git](https://git-scm.com/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command runs the test application locally, creates an Git repository via the AWS CodeCommit API locally, commits and pushes a test file to the repository, and then checks out the file in a fresh clone of the repository: -``` +## Run the application + +```bash make run ``` -You should then see a couple of log messages in the terminal: +The script: + +- Creates a CodeCommit Git repository via the CodeCommit API. +- Clones the repository to a temporary folder. +- Commits and pushes a test file to the repository. +- Clones the repository again to verify the committed file. + +You should then see log messages similar to: + ``` $ make run ----- @@ -41,19 +65,12 @@ Receiving objects: 100% (21/21), 1.55 KiB | 1.55 MiB/s, done. Step 3: Committing and pushing new file to the repository [master e7c599e] test_commit 1 file changed, 1 insertion(+) -Enumerating objects: 5, done. -Counting objects: 100% (5/5), done. -Delta compression using up to 8 threads -Compressing objects: 100% (2/2), done. -Writing objects: 100% (3/3), 292 bytes | 292.00 KiB/s, done. -Total 3 (delta 1), reused 0 (delta 0) +... To git://localhost:4510/repo1 7c1f7e8..e7c599e master -> master ----- Step 4: Cloning repo to second temporary folder Cloning into '/tmp/test.codecommit.repo1.copy'... -remote: counting objects: 24, done. -Receiving objects: 100% (24/24), 1.78 KiB | 608.00 KiB/s, done. ----- Step 5: Printing file content from second clone of repo test file content 123 diff --git a/cognito-jwt/Makefile b/cognito-jwt/Makefile index f39c8275..ad2e9925 100644 --- a/cognito-jwt/Makefile +++ b/cognito-jwt/Makefile @@ -6,6 +6,13 @@ export SERVICES = cognito usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -17,3 +24,9 @@ test-ci: # skip cognito tests in CI for now, as they require interactive input .PHONY: usage install run test-ci + + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d + diff --git a/cognito-jwt/README.md b/cognito-jwt/README.md index 8b7f7271..bb85a57b 100644 --- a/cognito-jwt/README.md +++ b/cognito-jwt/README.md @@ -1,56 +1,69 @@ -# LocalStack Demo: Cognito Auth with Email Verification +# Cognito Auth with JWT -Simple demo application illustrating Cognito authentication and user pools running locally using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Cognito | +| Integrations | AWS CLI | +| Categories | Authentication; Serverless | + +## Introduction + +A demo application illustrating Cognito authentication and user pools running locally using LocalStack. The sample creates a Cognito user pool, registers a user, handles email verification, and demonstrates JWT-based authentication — all without connecting to AWS. + +To receive Cognito email verification codes locally, configure the following environment variables before starting LocalStack: + +- `SMTP_HOST`: hostname and port of your SMTP server (e.g., `host.docker.internal:2525`) +- `SMTP_USER`: SMTP username (optional) +- `SMTP_PASS`: SMTP password (optional) +- `SMTP_EMAIL`: email address used to send messages ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack - -This scenario requires access to an SMTP server to send Cognito emails (e.g., to send codes for account activation). Please make sure the following environment variables are configured properly: -* `SMTP_HOST`: SMTP host -* `SMTP_USER`: SMTP username -* `SMTP_PASS`: SMTP password -* `SMTP_EMAIL`: Email address under which the messages should be sent +## Start LocalStack -Make sure that LocalStack is started: +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start -``` - -## Running -Please make sure to configure the environment variable `$USER_EMAIL` with an email address that can be used to send account activation emails. +## Run the application -Run the scenario script with the Cognito commands as follows: -``` +```bash make run ``` -You should see some log outputs from the script. At some point, the script will ask you to enter the confirmation code that has been sent to your email address (note: the code is also printed in the LocalStack terminal): +The script runs an interactive Cognito authentication scenario. Make sure to configure the environment variable `$USER_EMAIL` with an email address before running. + +At certain points, the script will prompt you to enter a confirmation code that has been sent to your email address (the code is also printed in the LocalStack logs): + ``` Please check email inbox for ..., and enter the confirmation code below: ``` -The script will then also ask you to specify a password reset code that is sent to your email (and also printed in the LocalStack terminal): +The script will then also ask you to specify a password reset code that is sent to your email: + ``` Please check email inbox for ..., and enter the password reset code here: ``` -## Credits - -* Kudos to `@Jaystified`, [Kurusugawa Computer Inc.](https://kurusugawa.jp) who kindly provided the initial version of the testing script in `test.sh`. +> Credits: Kudos to `@Jaystified`, [Kurusugawa Computer Inc.](https://kurusugawa.jp) who kindly provided the initial version of the testing script in `test.sh`. ## License diff --git a/ec2-docker-instances/Makefile b/ec2-docker-instances/Makefile index 6e25ccde..73220991 100644 --- a/ec2-docker-instances/Makefile +++ b/ec2-docker-instances/Makefile @@ -1,3 +1,11 @@ + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + # Install requirements install: docker pull ubuntu:focal @@ -9,7 +17,8 @@ run: start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + EC2_VM_MANAGER=docker DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/ec2-docker-instances/README.md b/ec2-docker-instances/README.md index 34e907a5..2b021043 100644 --- a/ec2-docker-instances/README.md +++ b/ec2-docker-instances/README.md @@ -1,41 +1,57 @@ -# LocalStack Demo: EC2 instances with Docker backend +# EC2 Instances with Docker Backend -This examples demos LocalStack EC2 and SSM functionalities when using the Docker backend. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | EC2, SSM | +| Integrations | AWS CLI | +| Categories | Compute | -## Prerequisites +## Introduction + +A demo application illustrating LocalStack EC2 and SSM functionality using the Docker backend. The sample creates Docker-backed EC2 instances, sends commands via SSM, captures output, takes snapshots as AMIs, and terminates instances. -* LocalStack -* Docker -* `make` -* [`jq`](https://stedolan.github.io/jq/) +> Note: This demo downloads the Ubuntu Docker image (~100MB) on first run. -Note: This demo involves the download of the Ubuntu Docker image weighing about 100MB +## Prerequisites -## Installing +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [`jq`](https://stedolan.github.io/jq/) -To install the dependencies: +## Check prerequisites + +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Run LocalStack with following enviroment flags: -``` -LOCALSTACK_AUTH_TOKEN=... EC2_VM_MANAGER=docker DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -Run the demo: -``` +## Run the application + +```bash make run ``` -You will see various operations being performed in the logs, starting with -- the creation of a Docker-backed EC2 instance -- command being sent to this instance through SSM -- retrieval standard output of this command -- snapshoting the running instance into a new AMI -- termination of the instance +The script: + +- Creates a Docker-backed EC2 instance. +- Sends a command to the instance via SSM. +- Retrieves the standard output of the command. +- Snapshots the running instance into a new AMI. +- Terminates the instance. ## License diff --git a/ecs-ecr-container-app/Makefile b/ecs-ecr-container-app/Makefile index 84a1aec8..ab1c1e0f 100644 --- a/ecs-ecr-container-app/Makefile +++ b/ecs-ecr-container-app/Makefile @@ -6,6 +6,13 @@ DOCKER_CONTAINER = localstack_nginx_1_0 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -38,7 +45,8 @@ clean: ## stop and remove created containers @((docker stop $(DOCKER_CONTAINER) && docker rm $(DOCKER_CONTAINER))>/dev/null 2>/dev/null) ||: start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/ecs-ecr-container-app/README.md b/ecs-ecr-container-app/README.md index 56f852df..fe242c50 100644 --- a/ecs-ecr-container-app/README.md +++ b/ecs-ecr-container-app/README.md @@ -1,52 +1,65 @@ -# LocalStack Demo: ECS Container App +# ECS Container Application -Simple demo application illustrating ECS applications running locally using LocalStack. The application image is built using Docker and pushed to a local ECR registry. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | ECS, ECR | +| Integrations | AWS CLI, CloudFormation, Docker | +| Categories | Containers; Serverless | + +## Introduction + +A demo application illustrating ECS container applications running locally using LocalStack. The application image is built using Docker, pushed to a local ECR registry, and deployed via CloudFormation to an ECS cluster. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: -``` -make install +```bash +make check ``` -## App Details +## Installation -Please refer to the `templates/` folder for details about the CloudFormation templates for the ECS service. +```bash +make install +``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command builds and deploys the app locally via CloudFormation: +## Deploy the Application -``` +```bash make deploy ``` -Specifically, the script above runs the following steps: -1. Create a new ECR registry locally -2. Build the application Docker image (from the `nginx` base image) -3. Push the image to the ECR registry -4. Create the ECS cluster and infrastructure -5. Create and deploy the ECS application, which starts the container in you local Docker environment +The script: -You should see some logs and a success output in the terminal: -``` +- Creates a new ECR registry locally. +- Builds the application Docker image from the `nginx` base image. +- Pushes the image to the ECR registry. +- Creates an ECS cluster and supporting infrastructure via CloudFormation. +- Deploys and starts the ECS application container in your local Docker environment. + +Please refer to the `templates/` folder for details about the CloudFormation templates for the ECS service. + +You should see a success output in the terminal: + +```bash ... Sample app (nginx) successfully deployed. ``` -Finally, the test app (nginx) should be accessible under the URL http://localhost:45139/ . +The nginx test app is accessible at `http://localhost:45139/` after deployment. ## License diff --git a/elb-load-balancing/Makefile b/elb-load-balancing/Makefile index 7f428408..217a6eb0 100644 --- a/elb-load-balancing/Makefile +++ b/elb-load-balancing/Makefile @@ -5,6 +5,14 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @npm install @which serverless || npm install -g serverless @@ -23,7 +31,8 @@ run: ## Deploy the app locally and run a Lambda test invocation start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/elb-load-balancing/README.md b/elb-load-balancing/README.md index 469858aa..29adef01 100644 --- a/elb-load-balancing/README.md +++ b/elb-load-balancing/README.md @@ -1,35 +1,53 @@ -# LocalStack Demo: ELB Application Load Balancers +# ELB Application Load Balancers -Simple demo application illustrating ELBv2 Application Load Balancers using LocalStack, deployed via the Serverless framework. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | ELB, Lambda | +| Integrations | Serverless Framework | +| Categories | Networking; Serverless | + +## Introduction + +A demo application illustrating ELBv2 Application Load Balancers using LocalStack, deployed via the Serverless framework. The sample deploys Lambda functions behind an Application Load Balancer and demonstrates HTTP invocations through the ELB endpoints. ## Prerequisites -* LocalStack -* Docker -* Node.js / `npm` -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [Node.js](https://nodejs.org/en/download/) with `npm` +- [Serverless Framework](https://www.serverless.com/framework/docs/getting-started) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -Deploy the app locally and run an ELB Lambda test invocation: -``` +## Run the application + +```bash make run ``` -You should see some output with the deployment logs of the Serverless application, and finally two successful invocations of the ELB endpoints `/hello1` and `/hello2`: -``` +The script deploys the Serverless application and invokes the Lambda functions via ELB endpoints `/hello1` and `/hello2`. + +You should see output similar to: + +```bash > sls deploy --stage local ... Serverless app successfully deployed. Now trying to invoke the Lambda functions via ELB endpoint. diff --git a/emr-serverless-python-dependencies/Makefile b/emr-serverless-python-dependencies/Makefile index e60f435c..14d4311c 100644 --- a/emr-serverless-python-dependencies/Makefile +++ b/emr-serverless-python-dependencies/Makefile @@ -2,6 +2,16 @@ export AWS_ACCESS_KEY_ID ?= test export AWS_SECRET_ACCESS_KEY ?= test export AWS_DEFAULT_REGION = us-east-1 + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v terraform > /dev/null 2>&1 || { echo "Terraform is not installed. Please install Terraform and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + + init: terraform workspace new local & terraform workspace new aws & @@ -30,6 +40,11 @@ run-aws: terraform workspace select aws ./start_job.sh aws + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d + stop: docker compose down diff --git a/emr-serverless-python-dependencies/README.md b/emr-serverless-python-dependencies/README.md index c6bc292e..fc1aad99 100644 --- a/emr-serverless-python-dependencies/README.md +++ b/emr-serverless-python-dependencies/README.md @@ -1,69 +1,88 @@ -# EMR Serverless with Python dependencies +# EMR Serverless with Python Dependencies -[AWS has this example](https://github.com/aws-samples/emr-serverless-samples/tree/main/examples/pyspark/dependencies) of how to add python dependencies to an emr job. Unfortunately, the same pattern isn't currently possible on LocalStack. This here will serve as a example of how to implement a workaround to still be able to add your own dependencies and module to your emr Spark jobs +| Key | Value | +| ------------ | ----------------------------------- | +| Services | EMR Serverless, S3, IAM | +| Integrations | Terraform, AWS CLI | +| Categories | Analytics; Big Data; Spark | -## Requirements -- Make -- Terraform ~>1.9.1 -- [LocalStack](https://github.com/localstack/localstack) -- [awslocal](https://github.com/localstack/awscli-local) +## Introduction -## init +A demo application illustrating how to add Python dependencies to an EMR Serverless Spark job using LocalStack. This sample implements a workaround for mounting Python environments directly into the LocalStack container, enabling PySpark jobs with custom dependencies to run locally. -This will initialize your terraform and terraform workspaces +## Prerequisites -``` -make init +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Terraform](https://developer.hashicorp.com/terraform/downloads) ~> 1.9.1 + +## Check prerequisites + +```bash +make check ``` -## Build +## Installation -This will build the python dependencies for the Spark job. This is where the first difference with AWS happens, as we will not package it like we do for aws, but intead will save the environment to our project folder to mount it to Localstack countainer. +This initializes your Terraform workspaces: +```bash +make init ``` -# For LocalStack, we create a /pyspark_env folder + +Build the Python dependencies for the Spark job. For LocalStack, we create a `/pyspark_env` folder that is mounted into the LocalStack container (rather than packaging it as a tarball like in AWS): + +```bash +# For LocalStack: creates /pyspark_env folder make build -# For aws, we create pyspark_deps.tar.gz +# For AWS: creates pyspark_deps.tar.gz make build-aws ``` -## Deploy - -Creates the following resources -- iam role -- iam policy -- s3 bucket -- emr-serverless application +## Start LocalStack +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -# Starts localstack using docker-compose, and apply the terraform configuration. -LOCALSTACK_AUTH_TOKEN= make deploy -# apply terraform configuration to AWS +## Deploy the Application + +Creates the following resources via Terraform: IAM role, IAM policy, S3 bucket, and an EMR Serverless application. + +```bash +# Deploy to LocalStack (starts LocalStack via docker-compose and applies Terraform) +LOCALSTACK_AUTH_TOKEN=$LOCALSTACK_AUTH_TOKEN make deploy + +# Deploy to AWS make deploy-aws ``` -## Run job +## Run the application -We can finally run our spark job. Notice the differences in the `start_job.sh` for LocalStack and aws. For aws we add `spark.archives` to our configuration and reference the path for the environment as `environment/bin/python`. Whereas for LocalStack, we rely on the volume mounted on our container instead of the archives and are using the absolute path for the environment `/tmp/environment/bin/python`. +We can finally run our Spark job. Notice the difference in `start_job.sh` between LocalStack and AWS: for AWS, `spark.archives` references `environment/bin/python`; for LocalStack, we rely on the volume-mounted container and use the absolute path `/tmp/environment/bin/python`. -``` +```bash # LocalStack make run -# aws +# AWS make run-aws ``` -## Destroy - -Finally we can destroy the environment. We make sure to stop the application first. +## Destroy the application -``` +```bash # LocalStack make destroy -# aws +# AWS make destroy-aws -``` \ No newline at end of file +``` + +## License + +This code is available under the Apache 2.0 license. diff --git a/emr-serverless-sample/Makefile b/emr-serverless-sample/Makefile new file mode 100644 index 00000000..e7094d52 --- /dev/null +++ b/emr-serverless-sample/Makefile @@ -0,0 +1,58 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +S3_BUCKET ?= test-emr-bucket +JOB_ROLE_ARN ?= arn:aws:iam::000000000000:role/emr-serverless-job-role + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @command -v mvn > /dev/null 2>&1 || { echo "Maven is not installed. Please install Maven and try again."; exit 1; } + @echo "All required prerequisites are available." + +install: ## Build the Java application + @cd hello-world && mvn package -q + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Create S3 bucket and upload the JAR file + @awslocal s3 mb s3://$(S3_BUCKET) || true + @awslocal s3 cp hello-world/target/hello-world-1.0.jar s3://$(S3_BUCKET)/code/hello-world-1.0.jar + +run: ## Create and run an EMR Serverless job + $(eval APP_ID := $(shell awslocal emr-serverless create-application \ + --type SPARK \ + --name hello-world-demo \ + --release-label emr-6.9.0 \ + --query applicationId --output text)) + @echo "Application ID: $(APP_ID)" + @awslocal emr-serverless start-application --application-id $(APP_ID) + @awslocal emr-serverless start-job-run \ + --application-id $(APP_ID) \ + --execution-role-arn $(JOB_ROLE_ARN) \ + --job-driver '{"sparkSubmit": {"entryPoint": "s3://$(S3_BUCKET)/code/hello-world-1.0.jar"}}' + +test-ci: ## Run CI tests + make check start install ready deploy run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/emr-serverless-sample/README.md b/emr-serverless-sample/README.md new file mode 100644 index 00000000..de78b655 --- /dev/null +++ b/emr-serverless-sample/README.md @@ -0,0 +1,58 @@ +# EMR Serverless Sample + +| Key | Value | +| ------------ | ----------------------- | +| Services | EMR Serverless, S3 | +| Integrations | AWS CLI, Maven | +| Categories | Analytics; Big Data | + +## Introduction + +A demo application illustrating how to run a Java Spark job on EMR Serverless using LocalStack. The sample builds a Maven project, uploads the JAR to S3, and submits an EMR Serverless job. + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Java](https://openjdk.org/) and [Maven](https://maven.apache.org/) + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` + +## Deploy the Application + +```bash +make deploy +``` + +The script creates an S3 bucket and uploads the compiled JAR file. + +## Run the application + +```bash +make run +``` + +The script creates an EMR Serverless application, starts it, and submits a Spark job. + +## License + +This code is available under the Apache 2.0 license. diff --git a/emr-serverless-spark/Makefile b/emr-serverless-spark/Makefile new file mode 100644 index 00000000..bba14bc7 --- /dev/null +++ b/emr-serverless-spark/Makefile @@ -0,0 +1,58 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +S3_BUCKET ?= test-emr-spark-bucket +JOB_ROLE_ARN ?= arn:aws:iam::000000000000:role/emr-serverless-job-role + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @command -v mvn > /dev/null 2>&1 || { echo "Maven is not installed. Please install Maven and try again."; exit 1; } + @echo "All required prerequisites are available." + +install: ## Build the Java Spark application + @cd hello-world && mvn package -q + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Create S3 bucket and upload the JAR file + @awslocal s3 mb s3://$(S3_BUCKET) || true + @awslocal s3 cp java-demo-1.0.jar s3://$(S3_BUCKET)/code/java-spark/java-demo-1.0.jar + +run: ## Create and run an EMR Serverless Spark job + $(eval APP_ID := $(shell awslocal emr-serverless create-application \ + --type SPARK \ + --name serverless-java-demo \ + --release-label emr-6.9.0 \ + --query applicationId --output text)) + @echo "Application ID: $(APP_ID)" + @awslocal emr-serverless start-application --application-id $(APP_ID) + @awslocal emr-serverless start-job-run \ + --application-id $(APP_ID) \ + --execution-role-arn $(JOB_ROLE_ARN) \ + --job-driver '{"sparkSubmit": {"entryPoint": "s3://$(S3_BUCKET)/code/java-spark/java-demo-1.0.jar", "sparkSubmitParameters": "--class HelloWorld"}}' + +test-ci: ## Run CI tests + make check start install ready deploy run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/emr-serverless-spark/README.md b/emr-serverless-spark/README.md index 102d63cb..a913ca6b 100644 --- a/emr-serverless-spark/README.md +++ b/emr-serverless-spark/README.md @@ -1,18 +1,33 @@ -# Running EMR Serverless Jobs with Java +# EMR Serverless Spark Jobs with Java -We will run a Java Spark job on EMR Serverless using a simple Java "Hello World" example in this example. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | EMR Serverless, S3 | +| Integrations | AWS CLI, Maven | +| Categories | Analytics; Big Data; Spark | + +## Introduction + +A demo application illustrating how to run a Java Spark job on EMR Serverless using LocalStack. The sample builds a Maven project, uploads the JAR to S3, creates an EMR Serverless application, and submits a Spark job — all locally without connecting to AWS. ## Prerequisites -* LocalStack -* `aws` CLI -* Docker -* Java and Maven +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Java](https://openjdk.org/) and [Maven](https://maven.apache.org/) -## Installation +## Check prerequisites + +```bash +make check +``` + +## Configure a Custom AWS Profile (optional) + +If not using `awslocal`, add the following profile to `~/.aws/config`: -### Configuring a custom profile -Configure a custom profile to use with LocalStack. Add the following profile to your AWS configuration file (by default, this file is at ~/.aws/config): ```shell [profile localstack] region=us-east-1 @@ -20,107 +35,66 @@ output=json endpoint_url = http://localhost:4566 ``` -Add the following profile to your AWS credentials file (by default, this file is at ~/.aws/credentials): +And to `~/.aws/credentials`: + ```shell [localstack] aws_access_key_id=test aws_secret_access_key=test ``` -Before creating the EMR Serverless job, we need to create a JAR file containing the Java code. We have the `java-demo-1.0.jar` file in the current directory. Alternatively, you can create the JAR file yourself by following the steps below. - -```bash -cd hello-world -mvn package -``` +## Installation -Next, we need to create an S3 bucket to store the JAR file. To do this, run the following command: +Build the Java Spark application (`java-demo-1.0.jar` is included; you can also rebuild it): ```bash -cd .. -export S3_BUCKET=test -aws s3 mb s3://$S3_BUCKET +make install ``` -You can now copy the JAR file from your current directory to the S3 bucket: +## Start LocalStack ```bash -aws s3 cp hello-world/target/java-demo-1.0.jar s3://${S3_BUCKET}/code/java-spark/java-demo-1.0.jar +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Creating the EMR Serverless Job +## Deploy the Application -Specify the ARN for the EMR Serverless job with the following command: +Before creating the EMR Serverless job, we need to create a JAR file containing the Java code. We have the `java-demo-1.0.jar` file in the current directory. Alternatively, you can create the JAR file yourself by following the steps below. ```bash -export JOB_ROLE_ARN=arn:aws:iam::000000000000:role/emr-serverless-job-role +cd hello-world +mvn package ``` -We can now create an EMR Serverless application, which will run Spark 3.3.0. Run the following command: +Create an S3 bucket and upload the JAR file: ```bash -aws emr-serverless create-application \ - --type SPARK \ - --name serverless-java-demo \ - --release-label "emr-6.9.0" \ - --initial-capacity '{ - "DRIVER": { - "workerCount": 1, - "workerConfiguration": { - "cpu": "4vCPU", - "memory": "16GB" - } - }, - "EXECUTOR": { - "workerCount": 3, - "workerConfiguration": { - "cpu": "4vCPU", - "memory": "16GB" - } - } - }' +cd .. +make deploy ``` -You can retrieve the Application ID from the output of the command, and export it as an environment variable: +## Run the application + +Creates an EMR Serverless Spark application (which will run Spark 3.3.0), starts it, and submits a job that runs the `HelloWorld` class: ```bash -export APPLICATION_ID='' +make run ``` -Start the EMR Serverless application: - -```shell -aws emr-serverless start-application \ - --application-id $APPLICATION_ID -``` +The Spark job submits with: +- Entry point: the JAR file in S3 +- `--class HelloWorld` +- Spark logs written to `s3:///logs/` (specified in the `logUri` parameter) -## Running the EMR Serverless Job +## Stop the application -You can now run the EMR Serverless job: +To stop the EMR Serverless application after the job completes: ```bash -aws emr-serverless start-job-run \ - --application-id $APPLICATION_ID \ - --execution-role-arn $JOB_ROLE_ARN \ - --job-driver '{ - "sparkSubmit": { - "entryPoint": "s3://'${S3_BUCKET}'/code/java-spark/java-demo-1.0.jar", - "sparkSubmitParameters": "--class HelloWorld" - } - }' \ - --configuration-overrides '{ - "monitoringConfiguration": { - "s3MonitoringConfiguration": { - "logUri": "s3://'${S3_BUCKET}'/logs/" - } - } - }' +awslocal emr-serverless stop-application --application-id $APPLICATION_ID ``` -The Spark logs will be written to the S3 bucket specified in the `logUri` parameter. You can stop the EMR Serverless application with the following command: +## License -```bash -aws emr-serverless stop-application \ - --application-id $APPLICATION_ID - -``` +This code is available under the Apache 2.0 license. diff --git a/glacier-s3-select/Makefile b/glacier-s3-select/Makefile index 2bdc3e54..82f6cf77 100644 --- a/glacier-s3-select/Makefile +++ b/glacier-s3-select/Makefile @@ -6,6 +6,13 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -15,7 +22,8 @@ run: ## Upload data and run Glacier/S3 Select queries locally start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/glacier-s3-select/README.md b/glacier-s3-select/README.md index c4dc1268..fccc6ae9 100644 --- a/glacier-s3-select/README.md +++ b/glacier-s3-select/README.md @@ -1,39 +1,57 @@ -# LocalStack Demo: Gacier and S3 Select Queries +# Glacier and S3 Select Queries -Simple demo application illustrating the use of Glacier API and S3 Select queries using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | S3, Glacier | +| Integrations | AWS CLI | +| Categories | Storage; Analytics | + +## Introduction + +A demo application illustrating the use of the Glacier API and S3 Select queries using LocalStack. The sample uploads CSV data, runs S3 Select queries to aggregate results, and demonstrates Glacier vault operations with select jobs. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: -``` -make install +```bash +make check ``` -## App Details +## Installation -Please refer to the `test.csv` file and feel free to modify in order to see changes in the query results. +```bash +make install +``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command creates local S3 buckets and Glacier vaults, and runs simple demo queries over the `data.csv` CSV file: +## Run the application -``` +```bash make run ``` -After the test script completes, the logs in your terminal should look similar to the output below: +The script: + +- Creates S3 buckets, uploads CSV data, and runs S3 Select queries. +- Creates a Glacier vault and uploads a CSV file. +- Initiates a Glacier select job and downloads the query results. + +Please refer to the `test.csv` file and feel free to modify it to see changes in the query results. + +You should see output similar to: + ``` $ make run Creating S3 bucket and Glacier vault in LocalStack diff --git a/glue-etl-jobs/Makefile b/glue-etl-jobs/Makefile index fe2952ac..3b92d92c 100644 --- a/glue-etl-jobs/Makefile +++ b/glue-etl-jobs/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -13,7 +20,8 @@ run: ## Prepare environment and run sample Glue PySpark ETL job ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/glue-etl-jobs/README.md b/glue-etl-jobs/README.md index d6938c12..bf26a50a 100644 --- a/glue-etl-jobs/README.md +++ b/glue-etl-jobs/README.md @@ -1,44 +1,58 @@ -# LocalStack Demo: Data Processing with Glue ETL Jobs +# Glue ETL Jobs -Simple demo application illustrating the use of the Glue API to run local ETL jobs using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Glue, S3, RDS | +| Integrations | AWS CLI | +| Categories | ETL; Analytics | + +## Introduction + +A demo application illustrating the use of the AWS Glue API to run local ETL (Extract, Transform, Load) jobs using LocalStack. The sample uploads a PySpark job script to S3, creates Glue databases and tables, and runs a Glue job to process data. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: -``` -make install +```bash +make check ``` -## App Details +## Installation -Please refer to the `job.py` PySpark job file and the `run.sh` script that runs the sample app. +```bash +make install +``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command prepares, creates, and runs the Glue job: +## Run the application -``` +```bash make run ``` -After the run script completes, the logs in your terminal should look similar to the output below: +The script uploads the PySpark job to S3, creates Glue databases and tables, starts the Glue job, and waits for it to complete. + +Please refer to the `job.py` PySpark job file and the `run.sh` script for implementation details. + +You should see output similar to: + ``` $ make run Putting PySpark script to test S3 bucket ... make_bucket: glue-pyspark-test -upload: ./job.py to s3://glue-pyspark-test/job.py +upload: ./job.py to s3://glue-pyspark-test/job.py Using local RDS database on port 4511 ... Creating Glue databases and tables ... Starting Glue job from PySpark script ... @@ -52,4 +66,4 @@ Done - Glue job execution finished. Please check the LocalStack container logs f ## License -This sample code is available under the Apache 2.0 license. +This code is available under the Apache 2.0 license. diff --git a/glue-msk-schema-registry/Makefile b/glue-msk-schema-registry/Makefile index b6d4a711..4394879b 100644 --- a/glue-msk-schema-registry/Makefile +++ b/glue-msk-schema-registry/Makefile @@ -5,6 +5,15 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v java > /dev/null 2>&1 || { echo "Java is not installed. Please install Java and try again."; exit 1; } + @command -v mvn > /dev/null 2>&1 || { echo "Maven is not installed. Please install Maven and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -16,7 +25,8 @@ run run-quiet: ## Prepare environment and run sample Glue MSK environment (no ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/glue-msk-schema-registry/README.md b/glue-msk-schema-registry/README.md index 34e2fa05..85e3c90c 100644 --- a/glue-msk-schema-registry/README.md +++ b/glue-msk-schema-registry/README.md @@ -1,36 +1,48 @@ -# LocalStack Demo: Schema Evolution with AWS Glue Schema Registry and AWS Managed Streaming for Kafka (MSK) +# Glue Schema Registry with MSK -Simple demo application illustrating the use of AWS MSK, Glue Schema Registry, Glue ETL, and RDS. +| Key | Value | +| ------------ | -------------------------------------------------------- | +| Services | MSK, Glue Schema Registry, Glue ETL, RDS | +| Integrations | AWS CLI, Maven | +| Categories | Streaming; Analytics; Schema Evolution | + +## Introduction + +A demo application illustrating schema evolution with AWS Glue Schema Registry and AWS Managed Streaming for Kafka (MSK) using LocalStack. The sample demonstrates Kafka producers registering and validating Avro schemas, Kafka consumers reading schema-encoded records, and the full schema evolution lifecycle with compatibility enforcement. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) -* Maven 3 -* Java 11 +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Java 11](https://openjdk.org/) and [Maven 3](https://maven.apache.org/) + +## Check prerequisites -## Installing -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -The install target only installs LocalStack and `awslocal`. -Please manually install Maven 3 and Java 11. -You can verify your installed versions by using Maven's version command: +The install target installs LocalStack and `awslocal`. Please also manually install Maven 3 and Java 11. Verify your versions: + ```sh $ mvn --version Apache Maven 3.6.3 Maven home: /usr/share/maven Java version: 11.0.15, vendor: Private Build, runtime: /usr/lib/jvm/java-11-openjdk-amd64 -Default locale: en_US, platform encoding: UTF-8 -OS name: "linux", version: "5.13.0-41-generic", arch: "amd64", family: "unix" ``` ## App Details -This example shows how to use the AWS Glue Schema Registry in combination with AWS Managed Streaming for Kafka (MSK) in LocalStack. + +This example shows how to use the AWS Glue Schema Registry in combination with MSK. + ```mermaid graph TD; Producer[Kafka Producer]-->|"(1) Validate / register schema"|Glue[AWS Glue Schema Registry] @@ -39,43 +51,43 @@ graph TD; Consumer[Kafka Consumer]---|"(4) Request schema (if not cached)"|Glue ``` -The example first performs a very basic workflow: -- Create a new Kafka Cluster using AWS MSK. -- Create a new AWS Glue Schema Registry. -- Create a new AWS Glue Schema with compatibility `BACKWARD`. -- Execute a Kafka Producer (`producer`) which sends 100 records compliant to the registered schema. - - The producer will validate that its schema is already registered in the registry. (1) - - The producer will compress the records using the schema and send them to the Kafka cluster. (2) -- Execute a Kafka Consumer (`consumer`) which waits for 100 records and processes them. - - The consumer receives the records (3). - - The consumer requests the schema from the registry if it's not already cached. (4) - -Afterwards, a more advanced schema evolution showcase is performed: -- Execute a second Kafka Producer (`producer-2`) which automatically registers its new schema version. - - This is successful, since the new version of the schema is `BACKWARD` compatible to the previous version. -- Use the AWS Glue Schema Registry to get a jsonpatch-diff of the two different schema versions. -- Execute the (previously already executed) Kafka Consumer (`consumer`), which now is incompatible to the new schema version. - - The execution _fails_ with a specific error message indicating that the schema of the record is not compatible. -- Execute a third Kafka Producer (`producer-3`) which tries to register another version of the schema. - - This time the registration _fails_, since the new version of the schema is _not_ `BACKWARD` compatible to the previous version. -- Finally, a second Kafka Consumer (`consumer-2`) is executed. - - This consumer's schema is compatible to the schema which was used by `producer-2` and the execution is successful. - -## Running +The example first performs a basic workflow: +- Create a Kafka Cluster (MSK), a Glue Schema Registry, and a Schema with `BACKWARD` compatibility. +- A Kafka Producer (`producer`) sends 100 Avro-encoded records, validating against the registered schema (1) and publishing to Kafka (2). +- A Kafka Consumer (`consumer`) reads the records (3) and fetches the schema from the registry if not cached (4). + +Then a more advanced schema evolution showcase: +- A second producer (`producer-2`) registers a new backward-compatible schema version. +- The registry returns a jsonpatch-diff of the two schema versions. +- Running the original consumer (`consumer`) against records from `producer-2` _fails_ — the schemas are incompatible. +- A third producer (`producer-3`) tries to register a schema that is _not_ backward-compatible — registration _fails_. +- A second consumer (`consumer-2`) compatible with `producer-2`'s schema runs successfully. + +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` + +## Run the application + ### Interactive Mode -The example allows being executed in "interactive mode". -After each step, you will be asked to enter any key to proceed with the execution of the next step. + +After each step, you will be asked to press any key to proceed: + ```sh make run-interactive ``` ### Quiet Mode -If you just want to execute the complete example, execute the following command: -```sh + +Execute all steps without interruption: + +```bash make run ``` -The example will execute all commands without interruption. ## License -This sample code is available under the Apache 2.0 license. +This code is available under the Apache 2.0 license. diff --git a/glue-redshift-crawler/Makefile b/glue-redshift-crawler/Makefile index ce702b10..25ef6d99 100644 --- a/glue-redshift-crawler/Makefile +++ b/glue-redshift-crawler/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -13,7 +20,8 @@ run: ## Prepare environment and run sample ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/glue-redshift-crawler/README.md b/glue-redshift-crawler/README.md index 4920371c..80ee1f9f 100644 --- a/glue-redshift-crawler/README.md +++ b/glue-redshift-crawler/README.md @@ -1,44 +1,54 @@ -# LocalStack Demo: Glue Crawler RedShift Integration (JDBC) +# Glue Crawler with Redshift -Simple demo application illustrating the use of AWS Glue Crawler to populate the Glue +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Glue, Redshift | +| Integrations | AWS CLI | +| Categories | ETL; Analytics | + +## Introduction + +A demo application illustrating how to use AWS Glue Crawler to populate the Glue metadata store with the table schema of Redshift database tables using LocalStack. The sample demonstrates the full workflow from creating a Redshift cluster to running a crawler that discovers table metadata. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) + +## Check prerequisites -## Installing -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## App Details -This example shows how to use AWS Glue Crawler to populate the Glue metadata store with the table schema of RedShift database tables. +## Start LocalStack -The following steps are executed when running the sample: -- Create a RedShift cluster and database. -- Create a Glue connection, specifying the JDBC connection properties for the RedShift database. -- Create a Glue database to store the table metadata in. -- Create a Crawler to populate the Glue database with the RedShift table metadata using the Glue connection. -- Create a new table in the RedShift database. -- Run the Crawler. -- Check out the resulting table metadata. - -## Running -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command executes the sample: +## Run the application -``` +```bash make run ``` +The script: + +- Creates a Redshift cluster and database. +- Creates a Glue connection with JDBC properties for the Redshift database. +- Creates a Glue database and a crawler. +- Runs the crawler to populate the Glue metadata store with Redshift table schemas. + ## License -This sample code is available under the Apache 2.0 license. +This code is available under the Apache 2.0 license. diff --git a/iam-policy-enforcement/Makefile b/iam-policy-enforcement/Makefile index 11309d3a..edc5298f 100644 --- a/iam-policy-enforcement/Makefile +++ b/iam-policy-enforcement/Makefile @@ -6,6 +6,13 @@ export ENFORCE_IAM=1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -15,7 +22,8 @@ run: ## Run the IAM tests locally @./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/iam-policy-enforcement/README.md b/iam-policy-enforcement/README.md index 168887e0..0c136f8d 100644 --- a/iam-policy-enforcement/README.md +++ b/iam-policy-enforcement/README.md @@ -1,37 +1,59 @@ -# LocalStack Demo: IAM Policy Enforcement +# IAM Policy Enforcement -Simple demo application illustrating enforcement of IAM policies when working with local cloud APIs in LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | IAM, S3, Kinesis | +| Integrations | AWS CLI | +| Categories | Security | + +## Introduction + +A demo application illustrating enforcement of IAM policies when working with local cloud APIs in LocalStack. The sample creates IAM users with specific policies and demonstrates allowed and denied API calls based on the configured permissions. + +> Note: IAM enforcement is not enabled by default. Set `ENFORCE_IAM=1` before starting LocalStack to enable it. ## Prerequisites -* LocalStack -* Docker -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Configuration - -Please note that LocalStack by default does not enforce IAM policies. IAM needs to be manually enabled by setting the `ENFORCE_IAM=1` environment variable. +## Start LocalStack -## Running +The Makefile exports `ENFORCE_IAM=1` automatically. Start LocalStack with: -Make sure that the `ENFORCE_IAM=1` environment variable is enabled, and that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... ENFORCE_IAM=1 DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -Run the script that is running the : -``` +## Run the application + +```bash make run ``` -You should see a couple of allowed and denied API calls (as per the IAM policies) in the terminal output: +The script demonstrates: + +- Denied Kinesis and S3 operations for users without the required IAM policies. +- Creating an IAM user with a policy that allows Kinesis access. +- Allowed Kinesis and S3 operations using the IAM credentials with the correct policy. + +You should see output similar to: + ``` Running IAM enforcement tests in local environment Step 1: Trying to create Kinesis stream - should get DENIED ... diff --git a/iot-basics/Makefile b/iot-basics/Makefile index ccd8d942..c6577b99 100644 --- a/iot-basics/Makefile +++ b/iot-basics/Makefile @@ -5,6 +5,14 @@ export AWS_DEFAULT_REGION=us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -26,7 +34,8 @@ run: ## Deploy the app locally make test-mqtt start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/iot-basics/README.md b/iot-basics/README.md index 95f141e0..1bce2ea4 100644 --- a/iot-basics/README.md +++ b/iot-basics/README.md @@ -1,33 +1,51 @@ -# LocalStack Demo: Basic IoT Entities +# IoT Basics -Simple demo application illustrating usage of IoT APIs in LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | IoT | +| Integrations | AWS CLI | +| Categories | IoT | + +## Introduction + +A demo application illustrating basic IoT API usage with LocalStack. The sample creates IoT things, policies, and certificates, then connects to the IoT MQTT endpoint to publish and subscribe to messages — all locally without connecting to AWS. ## Prerequisites -* LocalStack -* Docker -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -Deploy the app locally and run a Lambda test invocation: -``` +## Run the application + +```bash make run ``` -You should see a couple of successful API call outputs in the terminal: +The script creates IoT things and policies, connects to the local MQTT endpoint, publishes messages, and verifies they are received by a subscriber. + +You should see IoT API call outputs: + ``` { "things": [ @@ -52,7 +70,8 @@ You should see a couple of successful API call outputs in the terminal: ... ``` -The example then also connects to the IoT MQTT endpoint and sends/receives a couple of messages, see the exemplary output below: +The example then connects to the IoT MQTT endpoint and sends/receives messages: + ``` Running MQTT publish/subscribe test 10 messages published diff --git a/java-notification-app/Makefile b/java-notification-app/Makefile new file mode 100644 index 00000000..4550dfec --- /dev/null +++ b/java-notification-app/Makefile @@ -0,0 +1,51 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v mvn > /dev/null 2>&1 || { echo "Maven is not installed. Please install Maven and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + +install: ## Build the Java application + @mvn clean install -q + +start: ## Start LocalStack and supporting services via Docker Compose + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} docker-compose up -d + +stop: ## Stop all services + @echo + docker-compose down + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Deploy CloudFormation stack for notification infrastructure + @awslocal cloudformation deploy \ + --template-file src/main/resources/email-infra.yml \ + --stack-name email-infra + +run: ## Start the Spring Boot application and run a test notification + @awslocal ses verify-email-identity --email-address no-reply@localstack.cloud + AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test mvn spring-boot:run & + @sleep 5 + @awslocal sns publish \ + --topic arn:aws:sns:us-east-1:000000000000:email-notifications \ + --message '{"subject":"hello", "address": "alice@example.com", "body": "hello world"}' + @curl -s localhost:8080/process + +test-ci: ## Run CI tests + make check start install ready deploy run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/java-notification-app/README.md b/java-notification-app/README.md index dd0a1deb..6412b66e 100644 --- a/java-notification-app/README.md +++ b/java-notification-app/README.md @@ -1,70 +1,88 @@ -AWS Messaging: Spring Boot on LocalStack -======================================== +# Java Notification App -This sample Spring Boot application project demonstrates how to: +| Key | Value | +| ------------ | ----------------------------------- | +| Services | CloudFormation, SNS, SQS, SES | +| Integrations | AWS SDK, Docker Compose | +| Categories | Messaging; Notifications | -* Provision CloudFormation infrastructure on LocalStack -* Configure SNS SQS subscriptions with CloudFormation -* Receive SQS messages with the AWS Java SDK -* Send SES message with the AWS Java SDK +## Introduction -## Requirements +A Spring Boot application demonstrating AWS messaging services with LocalStack. The sample provisions CloudFormation infrastructure for SNS/SQS subscriptions, processes messages from SQS using the AWS Java SDK, and sends email notifications via SES. A MailHog SMTP server runs alongside LocalStack for local email testing. -* Java 11+ -* Maven 3+ -* [LocalStack](https://github.com/localstack/localstack) -* [awslocal](https://github.com/localstack/awscli-local) +## Prerequisites -## How To +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) and Docker Compose +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Java 11+](https://openjdk.org/) and [Maven 3+](https://maven.apache.org/) -### Build the application +## Check prerequisites -The application is a simple Spring Boot application that you can build by running +```bash +make check +``` - mvn clean install +## Installation -### Spin up the infrastructure on localstack +```bash +make install +``` -Resources are deployed via a CloudFormation template in `src/main/resources/email-infra.yml`. +This will install the Maven dependencies for the project and build the project. -First, start LocalStack and the SMTP server with: +## Start LocalStack - LOCALSTACK_AUTH_TOKEN= docker-compose up -d +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` -Then deploy the cloudformation stack (can take a few seconds) +This starts LocalStack and a MailHog SMTP server via Docker Compose. Access the MailHog UI at `http://localhost:8025/`. - awslocal cloudformation deploy \ - --template-file src/main/resources/email-infra.yml \ - --stack-name email-infra +## Deploy the Application -### Start the Spring Boot application +```bash +make deploy +``` -You can use `mvn spring-boot:run` to start the application, but you will need to set dummy AWS access credentials as environment variables: +Deploys the CloudFormation stack with SNS, SQS, and SES infrastructure. - AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test mvn spring-boot:run +## Run the application -### Test the application +Start the Spring Boot application (requires dummy AWS credentials as environment variables): -Verify the sender email address configured in the app +```bash +AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test mvn spring-boot:run +``` - awslocal ses verify-email-identity --email-address no-reply@localstack.cloud +### Test the application -Send a message to the topic +Run everything via `make run` or follow the steps below. - awslocal sns publish \ - --topic arn:aws:sns:us-east-1:000000000000:email-notifications \ - --message '{"subject":"hello", "address": "alice@example.com", "body": "hello world"}' +Verify the sender email address configured in the app: -Check the `/list` endpoint for queued messages. +```bash +awslocal ses verify-email-identity --email-address no-reply@localstack.cloud +``` - curl -s localhost:8080/list | jq . +Send a message to the topic: +```bash +awslocal sns publish --topic arn:aws:sns:us-east-1:000000000000:email-notifications --message '{"subject":"hello", "address": "alice@example.com", "body": "hello world"}' +``` -Run the `/process` endpoint to send the queued notifications as emails +Run the `/process` endpoint to send the queued notifications as emails: - curl -s localhost:8080/process +```bash +curl -s localhost:8080/process +``` Verify that the email has been sent: -* either check MailHog via the UI http://localhost:8025/ -* or query the LocalStack internal SES endpoint: `curl -s localhost:4566/_localstack/ses | jq .` +- either check MailHog via the UI http://localhost:8025/ +- or query the LocalStack internal SES endpoint: `curl -s localhost:4566/_localstack/ses | jq .` + +## License + +This code is available under the Apache 2.0 license. diff --git a/lambda-container-image/Makefile b/lambda-container-image/Makefile index ca403433..8a519108 100644 --- a/lambda-container-image/Makefile +++ b/lambda-container-image/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which awslocal || pip install awscli-local @@ -26,7 +33,8 @@ run: ## Build, deploy, and invoke the Lambda container image locally echo "Done - test successfully finished." start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/lambda-container-image/README.md b/lambda-container-image/README.md index d71ed76e..859ac523 100644 --- a/lambda-container-image/README.md +++ b/lambda-container-image/README.md @@ -1,37 +1,57 @@ -# LocalStack Demo: Lambda Container Images +# Lambda Container Images -Simple demo application illustrating Lambda container images in LocalStack. The Lambda image is built using Docker and pushed to a local ECR registry. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda, ECR | +| Integrations | AWS CLI, Docker | +| Categories | Serverless; Containers | + +## Introduction + +A demo application illustrating Lambda container images with LocalStack. The Lambda image is built using Docker and pushed to a local ECR registry, then deployed and invoked as a container-based Lambda function. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command builds, deploys, and runs the Lambda container image locally: +## Run the application -``` +```bash make run ``` -You should see some logs and a success output in the terminal: -``` -$ make run +The script: + +- Creates a local ECR repository. +- Builds the Lambda Docker image and pushes it to the ECR registry. +- Deploys the Lambda function using the container image. +- Invokes the Lambda function and shows the response. + +You should see output similar to: + +```bash Creating a new ECR repository locally Building the Docker image, pushing it to ECR URL: localhost:4513/repo1 ... @@ -50,14 +70,13 @@ Invoking Lambda function from container image Done - test successfully finished. ``` -The logs of the Lambda invocation should be visible in the LocalStack container output (with `DEBUG=1` enabled): -``` +The Lambda invocation logs are visible in the LocalStack container output (with `DEBUG=1` enabled): + +```bash DEBUG:localstack_ext.services.awslambda.lambda_extended: Log output for invocation of Lambda "ls-lambda-image": INIT: Using Lambda API Runtime target host: 'ls-lambda-image.us-east-1.localhost.localstack.cloud:4566' INIT: Starting daemons... INIT: Host 'ls-lambda-image.us-east-1.localhost.localstack.cloud' resolves to '172.17.0.2' -Starting XRay server loop on UDP port 2000 -Starting DNS server loop on UDP port 53 ----- Hello from LocalStack Lambda container image! ``` @@ -66,14 +85,17 @@ Hello from LocalStack Lambda container image! ### UnsupportedMediaTypeException -```plain +```bash An error occurred (UnsupportedMediaTypeException) when calling the Invoke operation (reached max retries: 0): The payload is not JSON: b'\xb5\xeb-\xb5\xeb-' ``` -**Solution**: Downgrade your awslocal CLI to version 1 because invoking lambda functions differs in version 2 (see [major changes in aws cli](https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration-changes.html) and limitations of [awscli-local](https://github.com/localstack/awscli-local)) or update the lambda invocation: +**Solution**: Downgrade your `awslocal` CLI to version 1, or update the Lambda invocation to use `--cli-binary-format raw-in-base64-out`: ```bash -awslocal lambda invoke --cli-binary-format raw-in-base64-out --function-name ls-lambda-image --payload '{"test": "test"}' /tmp/lambda.out --log-type Tail --query 'LogResult' --output text | base64 -d +awslocal lambda invoke --cli-binary-format raw-in-base64-out \ + --function-name ls-lambda-image \ + --payload '{"test": "test"}' /tmp/lambda.out \ + --log-type Tail --query 'LogResult' --output text | base64 -d ``` ## License diff --git a/lambda-debugging-sam-java/Makefile b/lambda-debugging-sam-java/Makefile index 1f2707c6..5b09b79f 100644 --- a/lambda-debugging-sam-java/Makefile +++ b/lambda-debugging-sam-java/Makefile @@ -23,6 +23,14 @@ all: usage usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v java > /dev/null 2>&1 || { echo "Java is not installed. Please install Java and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + # TODO: figure out how to install the AWS CLI v2 in CI install: ## Install CLI dependencies @which aws || pip install awscli @@ -69,6 +77,7 @@ clean: ## Clean the build directory rm -rf HelloWorldFunction/build start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) IMAGE_NAME=localstack/localstack-pro \ LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ diff --git a/lambda-debugging-sam-java/README.md b/lambda-debugging-sam-java/README.md index 733562c2..02d94f10 100644 --- a/lambda-debugging-sam-java/README.md +++ b/lambda-debugging-sam-java/README.md @@ -1,33 +1,46 @@ -# Debug your Java Lambda Function +# Lambda Remote Debugging (Java) -This Hello World Lambda function written in Java demonstrates how to use interactive breakpoint-debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). -LocalStack automatically configures debugging and adjusts relevant timeouts. +| Key | Value | +| ------------ | ------------------------------------------------------- | +| Services | Lambda | +| Integrations | AWS SAM, VS Code, AWS Toolkit for VS Code | +| Categories | Serverless; Debugging | -We recommend the one-click setup using the AWS Toolkit for VS Code unless your advanced scenario requires Lambda Debug Mode (Preview). +## Introduction -> [!NOTE] -> Check out our blog post [Developing with LocalStack using the AWS Toolkit for VS Code](#TODO-update-link) +A Hello World Java Lambda function demonstrating interactive breakpoint debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). LocalStack automatically configures debugging and adjusts relevant timeouts. The recommended setup uses the AWS Toolkit for VS Code for one-click debugging. ## Prerequisites -* [Docker](https://www.docker.com/) -* [VS Code](https://code.visualstudio.com/) -* [LocalStack Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 installs [LocalStack](https://docs.localstack.cloud/aws/getting-started/installation/) ≥ 4.8 -* [AWS Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 -* `make` -* [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [VS Code](https://code.visualstudio.com/) with the [LocalStack Toolkit](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 and [AWS Toolkit](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 +- [Java](https://openjdk.org/) and [Maven](https://maven.apache.org/) +- [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) -## Starting Up +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack 1. Execute the VS Code command "LocalStack: Run Setup Wizard" using the LocalStack Toolkit 2. Start LocalStack by clicking on the LocalStack Toolkit status bar -## Deploying +## Deploy the Lambda function 1. Run `make build` 2. Run `make deploy` -## Debugging +## Debugging the Lambda function 1. Open the **Remote invoke configuration** in the AWS Toolkit 1. Open the AWS Toolkit extension @@ -38,30 +51,30 @@ We recommend the one-click setup using the AWS Toolkit for VS Code unless your a 4. Set a breakpoint in your handler file by clicking in the gutter-margin 5. Click the **Remote invoke** button to invoke the Lambda function -## Lambda Debug Mode (Preview, Pro) +## Lambda Debug Mode ### Starting Up 1. Start LocalStack with the following configuration: ```sh - LOCALSTACK_IMAGE_NAME=localstack/localstack-pro \ + LOCALSTACK_AUTH_TOKEN= \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml \ localstack start --volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml ``` - * `IMAGE_NAME=localstack/localstack-pro` ensures the Pro image is started + * `LOCALSTACK_AUTH_TOKEN=` is the authentication token for LocalStack * `LOCALSTACK_LAMBDA_DEBUG_MODE=1` adjusts timeouts * `LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml` points to the config file for Lambda debug mode allowing for advanced configuration. It maps the Lambda function `arn:aws:lambda:us-east-1:000000000000:function:HelloWorldFunctionJava` to port `5050`. * `--volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml` maps the Lambda debug configuration from the host into the LocalStack Docker container for hot-reloading configuration updates. -### Deploying +### Deploy the Lambda function 1. Run `make build` to build the Lambda ZIP package 2. Run `make deploy` to deploy the Lambda function -### Debugging +### Debug the Lambda function 1. Open the sample folder in VS Code to auto-detect `.vscode/launch.json` 2. Set a breakpoint in the handler file `HelloWorldFunction/src/main/java/helloworld/App.java` by clicking in the gutter-margin diff --git a/lambda-debugging-sam-javascript/Makefile b/lambda-debugging-sam-javascript/Makefile index e9f78a71..1719b2d6 100644 --- a/lambda-debugging-sam-javascript/Makefile +++ b/lambda-debugging-sam-javascript/Makefile @@ -23,6 +23,14 @@ all: usage usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + # TODO: figure out how to install the AWS CLI v2 in CI install: ## Install CLI dependencies @which aws || pip install awscli @@ -88,7 +96,7 @@ clean: ## Clean the build directory rm -rf hello-world/hello-world-javascript.zip start: ## Start LocalStack - IMAGE_NAME=localstack/localstack-pro \ + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml \ diff --git a/lambda-debugging-sam-javascript/README.md b/lambda-debugging-sam-javascript/README.md index e0e64d14..0b5bd9df 100644 --- a/lambda-debugging-sam-javascript/README.md +++ b/lambda-debugging-sam-javascript/README.md @@ -1,33 +1,46 @@ -# Debug your JavaScript Lambda Function +# Lambda Remote Debugging (JavaScript) -This Hello World Lambda function written in JavaScript demonstrates how to use interactive breakpoint-debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). -LocalStack automatically configures debugging and adjusts relevant timeouts. +| Key | Value | +| ------------ | ------------------------------------------------------- | +| Services | Lambda | +| Integrations | AWS SAM, VS Code, AWS Toolkit for VS Code | +| Categories | Serverless; Debugging | -We recommend the one-click setup using the AWS Toolkit for VS Code unless your advanced scenario requires Lambda Debug Mode (Preview). +## Introduction -> [!NOTE] -> Check out our blog post [Developing with LocalStack using the AWS Toolkit for VS Code](#TODO-update-link) +A Hello World JavaScript Lambda function demonstrating interactive breakpoint debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). LocalStack automatically configures debugging and adjusts relevant timeouts. The recommended setup uses the AWS Toolkit for VS Code for one-click debugging. ## Prerequisites -* [Docker](https://www.docker.com/) -* [VS Code](https://code.visualstudio.com/) -* [LocalStack Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 installs [LocalStack](https://docs.localstack.cloud/aws/getting-started/installation/) ≥ 4.8 -* [AWS Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 -* `make` -* [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [VS Code](https://code.visualstudio.com/) with the [LocalStack Toolkit](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 and [AWS Toolkit](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 +- [Node.js](https://nodejs.org/en/download/) with `npm` +- [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) -## Starting Up +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack 1. Execute the VS Code command "LocalStack: Run Setup Wizard" using the LocalStack Toolkit 2. Start LocalStack by clicking on the LocalStack Toolkit status bar -## Deploying +## Deploy the Lambda function 1. Run `make build` 2. Run `make deploy` -## Debugging +## Debug the Lambda function 1. Open the **Remote invoke configuration** in the AWS Toolkit 1. Open the AWS Toolkit extension @@ -38,30 +51,30 @@ We recommend the one-click setup using the AWS Toolkit for VS Code unless your a 4. Set a breakpoint in your handler file by clicking in the gutter-margin 5. Click the **Remote invoke** button to invoke the Lambda function -## Lambda Debug Mode (Preview, Pro) +## Lambda Debug Mode ### Starting Up 1. Start LocalStack with the following configuration: ```sh - LOCALSTACK_IMAGE_NAME=localstack/localstack-pro \ + LOCALSTACK_AUTH_TOKEN= \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml \ localstack start --volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml ``` - * `IMAGE_NAME=localstack/localstack-pro` ensures the Pro image is started + * `LOCALSTACK_AUTH_TOKEN=` is the authentication token for LocalStack * `LOCALSTACK_LAMBDA_DEBUG_MODE=1` adjusts timeouts * `LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml` points to the config file for Lambda debug mode allowing for advanced configuration. It maps the Lambda function `arn:aws:lambda:us-east-1:000000000000:function:HelloWorldFunctionJavaScript` to port `6050`. * `--volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml` maps the Lambda debug configuration from the host into the LocalStack Docker container for hot-reloading configuration updates. -### Deploying +### Deploy the Lambda function 1. Run `make build` to build the Lambda ZIP package 2. Run `make deploy` to deploy the Lambda function -### Debugging +### Debug the Lambda function 1. Open the sample folder in VS Code to auto-detect `.vscode/launch.json` 2. Set a breakpoint in the handler file `hello-world/app.mjs` by clicking in the gutter-margin diff --git a/lambda-debugging-sam-python/Makefile b/lambda-debugging-sam-python/Makefile index 2956e0f6..bc64b8d4 100644 --- a/lambda-debugging-sam-python/Makefile +++ b/lambda-debugging-sam-python/Makefile @@ -23,6 +23,14 @@ all: usage usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + # TODO: figure out how to install the AWS CLI v2 in CI install: ## Install CLI dependencies @which aws || pip install awscli @@ -88,6 +96,7 @@ clean: ## Clean the build directory rm -rf hello_world/hello_world_python.zip start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) IMAGE_NAME=localstack/localstack-pro \ LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ diff --git a/lambda-debugging-sam-python/README.md b/lambda-debugging-sam-python/README.md index fd3f9cbe..44126cd3 100644 --- a/lambda-debugging-sam-python/README.md +++ b/lambda-debugging-sam-python/README.md @@ -1,33 +1,46 @@ -# Debug your Python Lambda Function +# Lambda Remote Debugging (Python) -This Hello World Lambda function written in Python demonstrates how to use interactive breakpoint-debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). -LocalStack automatically configures debugging and adjusts relevant timeouts. +| Key | Value | +| ------------ | ------------------------------------------------------- | +| Services | Lambda | +| Integrations | AWS SAM, VS Code, AWS Toolkit for VS Code | +| Categories | Serverless; Debugging | -We recommend the one-click setup using the AWS Toolkit for VS Code unless your advanced scenario requires Lambda Debug Mode (Preview). +## Introduction -> [!NOTE] -> Check out our blog post [Developing with LocalStack using the AWS Toolkit for VS Code](#TODO-update-link) +A Hello World Python Lambda function demonstrating interactive breakpoint debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). LocalStack automatically configures debugging and adjusts relevant timeouts. The recommended setup uses the AWS Toolkit for VS Code for one-click debugging. ## Prerequisites -* [Docker](https://www.docker.com/) -* [VS Code](https://code.visualstudio.com/) -* [LocalStack Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 installs [LocalStack](https://docs.localstack.cloud/aws/getting-started/installation/) ≥ 4.8 -* [AWS Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 -* `make` -* [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [VS Code](https://code.visualstudio.com/) with the [LocalStack Toolkit](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 and [AWS Toolkit](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 +- [Python 3](https://www.python.org/downloads/) +- [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) -## Starting Up +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack 1. Execute the VS Code command "LocalStack: Run Setup Wizard" using the LocalStack Toolkit 2. Start LocalStack by clicking on the LocalStack Toolkit status bar -## Deploying +## Deploy the Lambda function 1. Run `make build` 2. Run `make deploy` -## Debugging +## Debug the Lambda function 1. Open the **Remote invoke configuration** in the AWS Toolkit 1. Open the AWS Toolkit extension @@ -38,30 +51,30 @@ We recommend the one-click setup using the AWS Toolkit for VS Code unless your a 4. Set a breakpoint in your handler file by clicking in the gutter-margin 5. Click the **Remote invoke** button to invoke the Lambda function -## Lambda Debug Mode (Preview, Pro) +## Lambda Debug Mode ### Starting Up 1. Start LocalStack with the following configuration: ```sh - LOCALSTACK_IMAGE_NAME=localstack/localstack-pro \ + LOCALSTACK_AUTH_TOKEN= \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml \ localstack start --volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml ``` - * `IMAGE_NAME=localstack/localstack-pro` ensures the Pro image is started + * `LOCALSTACK_AUTH_TOKEN=` is the authentication token for LocalStack * `LOCALSTACK_LAMBDA_DEBUG_MODE=1` adjusts timeouts * `LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml` points to the config file for Lambda debug mode allowing for advanced configuration. It maps the Lambda function `arn:aws:lambda:us-east-1:000000000000:function:HelloWorldFunctionPython` to port `8050`. * `--volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml` maps the Lambda debug configuration from the host into the LocalStack Docker container for hot-reloading configuration updates. -### Deploying +### Deploy the Lambda function 1. Run `make build` to build the Lambda ZIP package 2. Run `make deploy` to deploy the Lambda function -### Debugging +### Debug the Lambda function 1. Open the sample folder in VS Code to auto-detect `.vscode/launch.json` 2. Set a breakpoint in the handler file `hello_world/app.py` by clicking in the gutter-margin diff --git a/lambda-debugging-sam-typescript/Makefile b/lambda-debugging-sam-typescript/Makefile index 9e355695..ea2fd572 100644 --- a/lambda-debugging-sam-typescript/Makefile +++ b/lambda-debugging-sam-typescript/Makefile @@ -23,6 +23,14 @@ all: usage usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + # TODO: figure out how to install the AWS CLI v2 in CI install: ## Install CLI dependencies @which aws || pip install awscli @@ -74,6 +82,7 @@ clean: ## Clean the build directory cd hello-world && rm -rf ./dist ./node_modules ./coverage start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) IMAGE_NAME=localstack/localstack-pro \ LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ diff --git a/lambda-debugging-sam-typescript/README.md b/lambda-debugging-sam-typescript/README.md index cce79603..cfdb07f1 100644 --- a/lambda-debugging-sam-typescript/README.md +++ b/lambda-debugging-sam-typescript/README.md @@ -1,33 +1,46 @@ -# Debug your TypeScript Lambda Function +# Lambda Remote Debugging (TypeScript) -This Hello World Lambda function written in JavaScript demonstrates how to use interactive breakpoint-debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). -LocalStack automatically configures debugging and adjusts relevant timeouts. +| Key | Value | +| ------------ | ------------------------------------------------------- | +| Services | Lambda | +| Integrations | AWS SAM, VS Code, AWS Toolkit for VS Code | +| Categories | Serverless; Debugging | -We recommend the one-click setup using the AWS Toolkit for VS Code unless your advanced scenario requires Lambda Debug Mode (Preview). +## Introduction -> [!NOTE] -> Check out our blog post [Developing with LocalStack using the AWS Toolkit for VS Code](#TODO-update-link) +A Hello World TypeScript Lambda function demonstrating interactive breakpoint debugging using LocalStack [Lambda Remote Debugging](https://docs.localstack.cloud/aws/tooling/lambda-tools/remote-debugging/). LocalStack automatically configures debugging and adjusts relevant timeouts. The recommended setup uses the AWS Toolkit for VS Code for one-click debugging. ## Prerequisites -* [Docker](https://www.docker.com/) -* [VS Code](https://code.visualstudio.com/) -* [LocalStack Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 installs [LocalStack](https://docs.localstack.cloud/aws/getting-started/installation/) ≥ 4.8 -* [AWS Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 -* `make` -* [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [VS Code](https://code.visualstudio.com/) with the [LocalStack Toolkit](https://marketplace.visualstudio.com/items?itemName=localstack.localstack) ≥ 1.2 and [AWS Toolkit](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) ≥ 3.74 +- [Node.js](https://nodejs.org/en/download/) with `npm` +- [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` -## Starting Up +## Start LocalStack 1. Execute the VS Code command "LocalStack: Run Setup Wizard" using the LocalStack Toolkit 2. Start LocalStack by clicking on the LocalStack Toolkit status bar -## Deploying +## Deploy the Lambda function 1. Run `make build` 2. Run `make deploy` -## Debugging +## Debug the Lambda function 1. Open the **Remote invoke configuration** in the AWS Toolkit 1. Open the AWS Toolkit extension @@ -48,30 +61,30 @@ We recommend the one-click setup using the AWS Toolkit for VS Code unless your a > * `.aws-sam/build/HelloWorldFunction` using workspace `code localstack-pro-samples/lambda-debugging-sam-typescript` > * `lambda-debugging-sam-typescript/.aws-sam/build/HelloWorldFunction` using workspace `code localstack-pro-samples` -## Lambda Debug Mode (Preview, Pro) +## Lambda Debug Mode ### Starting Up 1. Start LocalStack with the following configuration: ```sh - LOCALSTACK_IMAGE_NAME=localstack/localstack-pro \ + LOCALSTACK_AUTH_TOKEN= \ LOCALSTACK_LAMBDA_DEBUG_MODE=1 \ LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml \ localstack start --volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml ``` - * `IMAGE_NAME=localstack/localstack-pro` ensures the Pro image is started + * `LOCALSTACK_AUTH_TOKEN=` is the authentication token for LocalStack * `LOCALSTACK_LAMBDA_DEBUG_MODE=1` adjusts timeouts * `LOCALSTACK_LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/lambda_debug_mode_config.yaml` points to the config file for Lambda debug mode allowing for advanced configuration. It maps the Lambda function `arn:aws:lambda:us-east-1:000000000000:function:HelloWorldFunctionTypeScript` to port `7050`. * `--volume $PWD/lambda_debug_mode_config.yaml:/tmp/lambda_debug_mode_config.yaml` maps the Lambda debug configuration from the host into the LocalStack Docker container for hot-reloading configuration updates. -### Deploying +### Deploy the Lambda function 1. Run `make build` to build the Lambda ZIP package 2. Run `make deploy` to deploy the Lambda function -### Debugging +### Debug the Lambda function 1. Open the sample folder in VS Code to auto-detect `.vscode/launch.json` a. If using SAM, ensure `localRoot` is set to `${workspaceFolder}/hello-world` diff --git a/lambda-event-filtering/Makefile b/lambda-event-filtering/Makefile new file mode 100644 index 00000000..da9a9fc2 --- /dev/null +++ b/lambda-event-filtering/Makefile @@ -0,0 +1,49 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v samlocal > /dev/null 2>&1 || { echo "samlocal is not installed. Please run: pip install aws-sam-cli-local"; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install dependencies + @npm install + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Deploy the SAM application to LocalStack + @samlocal build + @samlocal deploy --no-confirm-changeset --resolve-s3 --stack-name lambda-event-filtering + +run: ## Send a test SQS message to trigger the Lambda function + @awslocal sqs send-message \ + --queue-url http://localhost:4566/000000000000/MyQueue \ + --message-body '{"data": "A"}' \ + --delay-seconds 10 + +test-ci: ## Run CI tests + make check start install ready deploy run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/lambda-event-filtering/README.md b/lambda-event-filtering/README.md index 21f70172..4c83aa8a 100644 --- a/lambda-event-filtering/README.md +++ b/lambda-event-filtering/README.md @@ -1,54 +1,66 @@ -# AWS Lambda event filtering with DynamoDB and SQS +# Lambda Event Filtering with DynamoDB and SQS -Simple demo application illustrating AWS Lambda event source filtering with DynamoDB and SQS. For this demo, we will use AWS Serverless Application Model (SAM), and a thin LocalStack wrapper `samlocal` to create our infrastructure through SAM on LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda, DynamoDB, SQS | +| Integrations | AWS SAM | +| Categories | Serverless; Event-Driven | -## Prerequisites - -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) -* [`samlocal`](https://github.com/localstack/aws-sam-cli-local) -* NodeJS 14.x -* [`ulid`](https://www.npmjs.com/package/ulid) +## Introduction -## Installing +A demo application illustrating AWS Lambda event source filtering with DynamoDB and SQS using LocalStack. The sample uses AWS SAM and `samlocal` to deploy a Lambda function that is triggered only when SQS messages match specific filtering criteria defined on the DynamoDB stream. -Setup [Serverless Application Model (SAM)](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) and [AWS SAM CLI Local](https://github.com/localstack/aws-sam-cli-local) on your local machine. We also recommend using NodeJS 14.x alongside a [Node Version Manager](https://github.com/nvm-sh/nvm) to manage your NodeJS versions. +## Prerequisites +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [`samlocal`](https://github.com/localstack/aws-sam-cli-local) — install with `pip install aws-sam-cli-local` +- [Node.js](https://nodejs.org/en/download/) with `npm` and [`ulid`](https://www.npmjs.com/package/ulid) (`npm install --save ulid`) -Start LocalStack via: +## Check prerequisites -```sh -localstack start -d +```bash +make check ``` -## Deploy the application +## Installation -Let us first install the local dependencies: +```bash +make install +``` + +## Start LocalStack -```sh -npm install --save ulid +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -To setup the infrastructure on LocalStack, run: +## Deploy the Application + +To set up the infrastructure on LocalStack for the first time, use the guided deploy (prompts for a stack name and confirmation): -```sh +```bash samlocal deploy -g ``` -You will be prompted to enter a name for the stack. Use the default options for the prompts and fill `Y` (`Yes`) for the confirmation prompt. The stack will be created and the output will be printed to the console. +Use the default options for the prompts and enter `Y` for the confirmation. For subsequent deploys after making changes: -If you have made any changes to the application, you can update the stack by running: - -```sh -samlocal deploy +```bash +samlocal deploy ``` -After deploying you can send a SQS message to the queue and see the Lambda function being triggered: +## Run the application + +After deploying, send an SQS message to trigger the Lambda via the filter criteria: -```sh -awslocal sqs send-message --queue-url http://localhost:4566/000000000000/MyQueue --message-body "{ "data" : "A" }" --delay-seconds 10 +```bash +awslocal sqs send-message \ + --queue-url http://localhost:4566/000000000000/MyQueue \ + --message-body '{"data": "A"}' \ + --delay-seconds 10 ``` You will see a JSON output similar to the following: @@ -66,8 +78,10 @@ Using the SQS, we send a message body to the DynamoDB stream to match the specif ## Destroy the application -To destroy the infrastructure on LocalStack, run: - -```sh +```bash samlocal delete ``` + +## License + +This code is available under the Apache 2.0 license. diff --git a/lambda-function-urls-javascript/Makefile b/lambda-function-urls-javascript/Makefile index cefe6926..0c749f18 100644 --- a/lambda-function-urls-javascript/Makefile +++ b/lambda-function-urls-javascript/Makefile @@ -5,6 +5,15 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v terraform > /dev/null 2>&1 || { echo "Terraform is not installed. Please install Terraform and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which awslocal || pip install awscli-local @@ -13,7 +22,7 @@ run: ## Build, deploy, and invoke the Lambda function URL zip function.zip index.js awslocal lambda create-function \ --function-name localstack-lamba-url-example \ - --runtime nodejs14.x \ + --runtime nodejs20.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/cool-stacklifter @@ -23,6 +32,7 @@ run: ## Build, deploy, and invoke the Lambda function URL --auth-type NONE start: + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) localstack start -d stop: diff --git a/lambda-function-urls-javascript/README.md b/lambda-function-urls-javascript/README.md index 9b35174f..3dc61cfb 100644 --- a/lambda-function-urls-javascript/README.md +++ b/lambda-function-urls-javascript/README.md @@ -1,46 +1,60 @@ -# Creating a Lambda function with a function URL +# Lambda Function URLs with JavaScript -In this example, we will demonstrate how to create a Lambda function with a function URL. With the Function URL property, there is now a new way to call a Lambda Function via HTTP API call. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda | +| Integrations | AWS CLI, Terraform | +| Categories | Serverless; REST API | + +## Introduction + +A demo application illustrating how to create a JavaScript Lambda function with a function URL using LocalStack. Lambda function URLs provide a dedicated HTTP(S) endpoint for invoking a Lambda function directly. The sample also demonstrates deploying the same setup using Terraform. ## Prerequisites -* LocalStack -* Docker -* `awslocal` CLI -* Terraform +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Terraform](https://developer.hashicorp.com/terraform/downloads) (optional, for Terraform-based deployment) -## Starting up +## Check prerequisites -Start LocalStack via: +```bash +make check +``` -```sh -localstack start -d +## Installation + +```bash +make install ``` -Push the following command to deploy the Lambda function: +## Start LocalStack -```sh +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` + +## Run the application + +Deploy the Lambda function and create a function URL: + +```bash awslocal lambda create-function \ --function-name localstack-lamba-url-example \ - --runtime nodejs14.x \ + --runtime nodejs20.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/cool-stacklifter -``` - -## Creating a Lambda function URL - -With the Function URL property, there is now a new way to call a Lambda Function via HTTP API call using the `create-function-url-config` command. -```sh awslocal lambda create-function-url-config \ --function-name localstack-lamba-url-example \ --auth-type NONE ``` -You will retrieve a HTTP URL that you can use to invoke the Lambda function, in the form of `http://abcdefgh.lambda-url.us-east-1.localhost.localstack.cloud:4566`. - -You can now trigger the Lambda function by sending a HTTP POST request to the URL using `curl`: +You will receive an HTTP URL in the form `http://abcdefgh.lambda-url.us-east-1.localhost.localstack.cloud:4566`. Invoke the Lambda function via HTTP POST: ```sh curl -X POST \ @@ -49,20 +63,24 @@ curl -X POST \ -d '{"num1": "10", "num2": "10"}' ``` -The following output would be retrieved: +Expected output: -```sh -The product of 10 and 10 is 100% +``` +The product of 10 and 10 is 100 ``` ## Using Terraform -You can use Terraform to automate the creation of Lambda function and to create a function URL. Run the following commands on your terminal to create the Lambda function and the function URL: +You can automate the Lambda function and function URL creation using Terraform: ```sh -terraform init +terraform init terraform plan terraform apply --auto-approve ``` -Since we are using LocalStack, no actual AWS resources will be created. Instead, LocalStack will create ephemeral development resources, which will automatically be cleaned once you stop LocalStack (using `localstack stop`). +Since we are using LocalStack, no actual AWS resources will be created. LocalStack creates ephemeral development resources that are automatically cleaned up when you stop LocalStack (`localstack stop`). + +## License + +This code is available under the Apache 2.0 license. diff --git a/lambda-function-urls-javascript/main.tf b/lambda-function-urls-javascript/main.tf index 827832d6..58c21f04 100644 --- a/lambda-function-urls-javascript/main.tf +++ b/lambda-function-urls-javascript/main.tf @@ -6,7 +6,7 @@ resource "aws_lambda_function" "example_lambda" { role = "cool-stacklifter" handler = "index.handler" source_code_hash = filebase64sha256("function.zip") - runtime = "nodejs14.x" + runtime = "nodejs20.x" } resource "aws_lambda_function_url" "lambda_function_url" { diff --git a/lambda-function-urls-python/Makefile b/lambda-function-urls-python/Makefile index 76108bd5..887041cd 100644 --- a/lambda-function-urls-python/Makefile +++ b/lambda-function-urls-python/Makefile @@ -5,6 +5,15 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v terraform > /dev/null 2>&1 || { echo "Terraform is not installed. Please install Terraform and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which awslocal || pip install awscli-local @@ -25,6 +34,7 @@ run: ## Build, deploy, and invoke the Lambda function URL --auth-type NONE start: + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) localstack start -d stop: diff --git a/lambda-function-urls-python/README.md b/lambda-function-urls-python/README.md new file mode 100644 index 00000000..488d1038 --- /dev/null +++ b/lambda-function-urls-python/README.md @@ -0,0 +1,54 @@ +# Lambda Function URLs with Python + +| Key | Value | +| ------------ | ----------------------------- | +| Services | Lambda | +| Integrations | AWS CLI, Terraform | +| Categories | Serverless; REST API | + +## Introduction + +A demo application illustrating how to create a Python Lambda function with a function URL using LocalStack. Lambda function URLs provide a dedicated HTTP(S) endpoint for invoking a Lambda function directly without an API Gateway. + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` + +## Run the application + +```bash +make run +``` + +The script: + +- Packages the Lambda function with its dependencies. +- Creates the Lambda function in LocalStack. +- Configures a function URL with no authentication. + +## License + +This code is available under the Apache 2.0 license. diff --git a/lambda-hot-reloading/Makefile b/lambda-hot-reloading/Makefile new file mode 100644 index 00000000..d7a9215e --- /dev/null +++ b/lambda-hot-reloading/Makefile @@ -0,0 +1,55 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install dependencies for the JavaScript sample + @cd javascript && npm install + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +deploy: ## Deploy the JavaScript Lambda with hot reloading enabled + @cd javascript && \ + awslocal iam create-role \ + --role-name lambda-role \ + --assume-role-policy-document file://trust-policy.json 2>/dev/null || true && \ + awslocal lambda create-function \ + --function-name hot-reload-js \ + --runtime nodejs18.x \ + --role arn:aws:iam::000000000000:role/lambda-role \ + --code S3Bucket=hot-reload,S3Key=$$(pwd) \ + --handler index.handler + +run: ## Invoke the hot-reload Lambda function + @awslocal lambda invoke \ + --function-name hot-reload-js \ + --payload '{}' \ + output.json && cat output.json + +test-ci: ## Run CI tests + make check start install ready deploy run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs deploy run test-ci diff --git a/lambda-hot-reloading/README.md b/lambda-hot-reloading/README.md new file mode 100644 index 00000000..64b39a76 --- /dev/null +++ b/lambda-hot-reloading/README.md @@ -0,0 +1,66 @@ +# Lambda Hot Reloading + +| Key | Value | +| ------------ | ----------------------------- | +| Services | Lambda | +| Integrations | AWS CLI, Terraform | +| Categories | Serverless; Developer Experience | + +## Introduction + +A collection of samples demonstrating LocalStack's Lambda hot reloading capability, which enables immediate reflection of code changes without redeployment. Hot reloading significantly accelerates development cycles by watching local file changes and automatically updating Lambda function code. + +## Sub-samples + +| Sample | Description | +| ------------------------- | ------------------------------------------------- | +| [javascript](javascript) | Hot reloading with a JavaScript Lambda function | +| [typescript](typescript) | Hot reloading with a TypeScript Lambda function | +| [javascript-terraform](javascript-terraform) | Hot reloading deployed via Terraform | +| [javascript-terraform-layers](javascript-terraform-layers) | Hot reloading with Lambda layers via Terraform | +| [lambda-typescript-webpack](lambda-typescript-webpack) | TypeScript Lambda with Webpack bundling | + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Node.js](https://nodejs.org/en/download/) with `npm` + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` + +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` + +## Deploy the Application + +```bash +make deploy +``` + +## Run the application + +```bash +make run +``` + +Refer to the individual sub-sample READMEs for more details on each variant. + +## License + +This code is available under the Apache 2.0 license. diff --git a/lambda-hot-reloading/lambda-typescript-webpack/README.md b/lambda-hot-reloading/lambda-typescript-webpack/README.md index eea130ec..9fa90cf8 100644 --- a/lambda-hot-reloading/lambda-typescript-webpack/README.md +++ b/lambda-hot-reloading/lambda-typescript-webpack/README.md @@ -17,6 +17,7 @@ This is a simple example of how to hot reload your Typescript Lambda with Webpac Start your LocalStack Docker container with the following command: ```bash +export LOCALSTACK_AUTH_TOKEN= localstack start ``` diff --git a/lambda-php-bref-cdk-app/Makefile b/lambda-php-bref-cdk-app/Makefile index ba11d6e7..041d4c67 100644 --- a/lambda-php-bref-cdk-app/Makefile +++ b/lambda-php-bref-cdk-app/Makefile @@ -9,6 +9,14 @@ DOCKER_COMPOSER_IMAGE ?= composer:2.5 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @npm install @which cdklocal || npm install -g aws-cdk-local aws-cdk @@ -38,7 +46,8 @@ invoke: curl $${endpoint}?name=LocalStack! start: - PROVIDER_OVERRIDE_lambda=v2 localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/lambda-php-bref-cdk-app/README.md b/lambda-php-bref-cdk-app/README.md index 5af97e72..5ce97cd6 100644 --- a/lambda-php-bref-cdk-app/README.md +++ b/lambda-php-bref-cdk-app/README.md @@ -1,80 +1,50 @@ -# LocalStack Demo: Deploying PHP/Bref Lambda via CDK +# PHP/Bref Lambda with CDK -Simple [PHP/Bref](https://bref.sh/) serverless application using a shared Lambda layer deployable with AWS CDK to LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda, API Gateway | +| Integrations | AWS CDK, Bref, PHP | +| Categories | Serverless; PHP | -This PHP/Bref application **without fpm** implements a [typed PHP Lambda handler](https://bref.sh/docs/function/handlers.html) as an *HTTP handler class* for serving [API Gateway HTTP events](https://bref.sh/docs/function/handlers.html#api-gateway-http-events). -Bref turns an API Gateway event into a [PSR-7](https://www.php-fig.org/psr/psr-7/)request and one Lambda per route implements a handler class and returns a PSR-7 response. +## Introduction -## PHP/Bref with fpm and Serverless - -Bref typically runs [Web applications on AWS Lambda](https://bref.sh/docs/runtimes/http.html) to support traditional PHP frameworks such as Laravel and Symphony. -In this `php-fpm` approach, Bref turns an API Gateway event into a FastCGI (PHP-FPM) request and one Lambda receives all URLs and responds using `echo`, `header()` function, etc. -Checkout the different kinds of applications at [php-runtime/bref](https://github.com/php-runtime/bref) and select the correct layer with or without `fpm` accordingly. - -To deploy the `php-fpm` Laravel [base](https://github.com/brefphp/examples/tree/master/Laravel/base) project from [brefphp/examples](https://github.com/brefphp/examples) to LocalStack: - -1. Install the [serverless-localstack](https://github.com/LocalStack/serverless-localstack) plugin - - ```bash - npm install --save-dev serverless-localstack - ``` - -2. Add serverless-localstack to `plugins` in the [serverless.yml](https://github.com/brefphp/examples/blob/master/Laravel/base/serverless.yml) - - ```yml - plugins: - - ./vendor/bref/bref - - serverless-localstack - ``` - -3. Add `custom` properties in the `serverless.yml` - - ```yml - custom: - localstack: - # list of stages for which the plugin should be enabled - stages: - - local - ``` - -4. Deploy to LocalStack - - ```bash - serverless deploy --stage local - ``` +A [PHP/Bref](https://bref.sh/) serverless application using a shared Lambda layer deployable with AWS CDK to LocalStack. The application implements a typed PHP Lambda handler as an HTTP handler class for serving API Gateway HTTP events. Bref turns API Gateway events into PSR-7 requests for PHP processing. ## Prerequisites -* LocalStack -* Docker -* `make` -* `curl` -* `jq` -* Node.js / `npm` -* [`cdklocal`](https://github.com/localstack/aws-cdk-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`cdklocal`](https://github.com/localstack/aws-cdk-local) — install with `npm install -g aws-cdk-local` +- [Node.js](https://nodejs.org/en/download/) with `npm` +- [`curl`](https://curl.se/) and [`jq`](https://stedolan.github.io/jq/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=$LOCALSTACK_AUTH_TOKEN DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Deploy the app locally and run an HTTP test invocation: ```bash make run ``` -The script first bootstraps and deploys the CDK app locally and subsequently invokes the HTTP endpoint via curl (`make invoke`). +The script bootstraps and deploys the CDK app locally, then invokes the HTTP endpoint via `curl`. You should see output similar to: ``` Outputs: @@ -84,15 +54,40 @@ arn:aws:cloudformation:us-east-1:000000000000:stack/CdkBrefStack/dec480c5 ✨ Total time: 7.9s - CDK app successfully deployed. Now trying to invoke the Lambda through API gateway. -endpoint=$(jq .CdkBrefStack.Url cdk-outputs.json --raw-output) && \ - echo endpoint=${endpoint} && \ - curl ${endpoint}?name=LocalStack! endpoint=https://bd0f6b19.execute-api.localhost.localstack.cloud:4566/ -Hello LocalStack!% +Hello LocalStack! ``` +## PHP/Bref with FPM and Serverless Framework + +Bref also supports running web applications on AWS Lambda via [php-fpm](https://bref.sh/docs/runtimes/http.html), enabling traditional PHP frameworks like Laravel and Symfony. To deploy a `php-fpm` Laravel project from [brefphp/examples](https://github.com/brefphp/examples) to LocalStack: + +1. Install the [serverless-localstack](https://github.com/LocalStack/serverless-localstack) plugin: + ```bash + npm install --save-dev serverless-localstack + ``` + +2. Add `serverless-localstack` to `plugins` in `serverless.yml`: + ```yml + plugins: + - ./vendor/bref/bref + - serverless-localstack + ``` + +3. Add `custom` properties: + ```yml + custom: + localstack: + stages: + - local + ``` + +4. Deploy: + ```bash + serverless deploy --stage local + ``` + ## License This code is available under the Apache 2.0 license. diff --git a/lambda-xray/README.md b/lambda-xray/README.md new file mode 100644 index 00000000..53c32e58 --- /dev/null +++ b/lambda-xray/README.md @@ -0,0 +1,39 @@ +# Lambda X-Ray Tracing + +| Key | Value | +| ------------ | --------------------------------------------- | +| Services | Lambda, X-Ray, CloudFormation | +| Integrations | AWS CLI, AWS SAM | +| Categories | Serverless; Observability | + +## Introduction + +A collection of samples demonstrating AWS X-Ray distributed tracing for Lambda functions using LocalStack. The samples show how to instrument Python Lambda functions with the AWS X-Ray SDK and AWS Powertools Tracer, and how to retrieve and inspect trace data locally. + +## Sub-samples + +| Sample | Description | +| ------------------------- | -------------------------------------------------------- | +| [python-xray-sdk](python-xray-sdk) | Lambda X-Ray tracing using the AWS X-Ray SDK for Python | +| [python-powertools](python-powertools) | Lambda X-Ray tracing using AWS Powertools Tracer | + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) with `pip` + +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +localstack start -d +``` + +Refer to the individual sub-sample READMEs for deployment and invocation instructions. + +## License + +This code is available under the Apache 2.0 license. diff --git a/mediastore-uploads/Makefile b/mediastore-uploads/Makefile index 4b60cf54..eb7f46f7 100644 --- a/mediastore-uploads/Makefile +++ b/mediastore-uploads/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack > /dev/null || pip install localstack @which awslocal > /dev/null || pip install awscli-local @@ -13,7 +20,8 @@ run: ## Run tests uploading objects via MediaStore API ./test.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/mediastore-uploads/README.md b/mediastore-uploads/README.md index 2e0b37f3..0a869af5 100644 --- a/mediastore-uploads/README.md +++ b/mediastore-uploads/README.md @@ -1,35 +1,52 @@ -# LocalStack Demo: MediaStore Containers +# MediaStore Uploads -Simple demo application illustrating the use of the MediaStore API using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | MediaStore | +| Integrations | AWS CLI | +| Categories | Media; Storage | + +## Introduction + +A demo application illustrating the use of the AWS MediaStore API with LocalStack. The sample creates a MediaStore container, uploads files to it, downloads files from it, and cleans up the resources. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command runs the example, which creates a MediaStore container, uploads and downloads files to/from the container, and finally cleans up the created resources: -``` +## Run the application + +```bash make run ``` -After the test script completes, the logs in your terminal should look similar to the output below: -``` +The script creates a MediaStore container, uploads a test file, downloads it and verifies the content, then deletes the container. + +You should see output similar to: + +```bash $ make run Creating MediaStore container in LocalStack ... MediaStore container endpoint: http://localhost:4510/my-container1 diff --git a/mq-broker/Makefile b/mq-broker/Makefile index 69394b3f..daf171ff 100644 --- a/mq-broker/Makefile +++ b/mq-broker/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack > /dev/null || pip install localstack @which awslocal > /dev/null || pip install awscli-local @@ -13,7 +20,8 @@ run: ## Run test creating a broker and sending a message ./test.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/mq-broker/README.md b/mq-broker/README.md index 05bd54ca..2b23d16b 100644 --- a/mq-broker/README.md +++ b/mq-broker/README.md @@ -1,34 +1,51 @@ -# LocalStack Demo: MQ Broker +# MQ Broker -Simple demo application illustrating the use of MQ using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | MQ | +| Integrations | AWS CLI | +| Categories | Messaging | + +## Introduction + +A demo application illustrating the use of the AWS MQ API with LocalStack. The sample creates a message broker, connects to it, sends a message to a queue, and cleans up the broker. ## Prerequisites -* LocalStack -* Docker -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command runs the example, which starts up a broker and sends a message to a queue: -``` +## Run the application + +```bash make run ``` -After the test script completes, the logs in your terminal should look similar to the output below: +The script creates an MQ broker, retrieves the broker endpoint, sends a message to the queue, and deletes the broker. + +You should see output similar to: + ``` $ make run Creating MQ broker in LocalStack ... diff --git a/multi-account-multi-region-s3-access/Makefile b/multi-account-multi-region-s3-access/Makefile index d63acd8c..f84480be 100644 --- a/multi-account-multi-region-s3-access/Makefile +++ b/multi-account-multi-region-s3-access/Makefile @@ -7,6 +7,13 @@ SHELL := /bin/bash usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -15,6 +22,7 @@ run: ## Run the cross-account cross-region experiment of copying data fr ./run.sh start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) localstack start -d stop: ## Stop LocalStack diff --git a/multi-account-multi-region-s3-access/README.md b/multi-account-multi-region-s3-access/README.md index d5ff1611..e1c85751 100644 --- a/multi-account-multi-region-s3-access/README.md +++ b/multi-account-multi-region-s3-access/README.md @@ -1,48 +1,55 @@ -# Localstack Demo: Access S3 resources from different account and different region +# Multi-Account Multi-Region S3 Access -Simple demo script to showcase the accessing of S3 resources from a different AWS account using bucket policies and IAM users with specific IAM policies attached to their identities. -The script uses a couple of AWS profiles to achieve that: +| Key | Value | +| ------------ | ----------------------------------- | +| Services | S3, IAM | +| Integrations | AWS CLI | +| Categories | Security; Multi-Account | -* Admin user of account A with account ID `000000000001`. +## Introduction -* Admin user of account B with account ID `000000000002`. +A demo application illustrating how to access S3 resources across different AWS accounts using bucket policies and IAM users with LocalStack. The script uses the following AWS profiles to simulate a cross-account scenario: -* Account A user that creates the S3 bucket and subsequent resources inside the bucket. - -* Account B user that copies the resources from account A user's S3 bucket `source` into a bucket `target` it owns. +- **Admin user of account A** with account ID `000000000001` +- **Admin user of account B** with account ID `000000000002` +- **Account A user** — creates the S3 bucket and grants cross-account access via bucket policies +- **Account B user** — copies resources from Account A's `source` bucket into its own `target` bucket ## Prerequisites -* LocalStack -* Docker -* Python 3.6+ / Python Pip -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) with `pip` + +## Check prerequisites -## Installing +```bash +make check +``` -To install the dependencies: +## Installation -```shell +```bash make install ``` -## Starting LocalStack - -Make sure that LocalStack is started: +## Start LocalStack -```shell -LOCALSTACK_AUTH_TOKEN=... make start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Run the sample demo script: - -```shell +```bash make run ``` +The script sets up two AWS accounts with IAM users and bucket policies, then demonstrates cross-account S3 resource access. + ## License This code is available under the Apache 2.0 license. - diff --git a/neptune-graph-db/Makefile b/neptune-graph-db/Makefile index 45d1fc49..20174c58 100644 --- a/neptune-graph-db/Makefile +++ b/neptune-graph-db/Makefile @@ -7,6 +7,14 @@ PYTHON_BIN ?= $(shell which python3 || which python) usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -16,7 +24,8 @@ run: ## Run the scenario with queries against local Neptune @$(PYTHON_BIN) query.py start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/neptune-graph-db/README.md b/neptune-graph-db/README.md index 87d70023..6f07fbd2 100644 --- a/neptune-graph-db/README.md +++ b/neptune-graph-db/README.md @@ -1,37 +1,52 @@ -# LocalStack Demo: Neptune Graph Database +# Neptune Graph Database -Simple demo application illustrating the use of Neptune Graph DB queries locally, using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Neptune | +| Integrations | AWS CLI | +| Categories | Graph Database | + +## Introduction + +A demo application illustrating Neptune Graph DB queries running locally using LocalStack. The sample creates a Neptune cluster, connects to it via Gremlin, and performs graph operations — adding vertices and querying the graph — all without connecting to AWS. ## Prerequisites -* LocalStack (Pro version) -* Docker -* Python -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Run the scenario Python script `query.py` as follows: -``` +```bash make run ``` -You should see some logs from the script, similar to the output below: +The script creates a Neptune cluster, connects via Gremlin WebSocket, submits values, adds vertices to the graph, and queries the results. + +You should see output similar to: + ``` Creating Neptune Graph DB cluster "cluster123" - this may take a few moments ... Connecting to Neptune Graph DB cluster URL: ws://localhost:4510/gremlin @@ -42,7 +57,9 @@ Adding new vertices "v1" and "v2" to the graph New list of vertices in the graph: [v[0], v[3]] Deleting Neptune Graph DB cluster "cluster123" ``` -_Note: when running the scenario consecutively, it may takes some time to free the port used by Neptune._ + +> Note: When running the scenario consecutively, it may take some time to free the port used by Neptune. + ## License -The code in this sample repo is available under the Apache 2.0 license. +This code is available under the Apache 2.0 license. diff --git a/qldb-ledger-queries/Makefile b/qldb-ledger-queries/Makefile index 06d97edd..c56ec566 100644 --- a/qldb-ledger-queries/Makefile +++ b/qldb-ledger-queries/Makefile @@ -6,6 +6,14 @@ PYTHON_BIN ?= $(shell which python3 || which python) usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -15,7 +23,8 @@ run: ## Run the sample queries against the local QLDB API @$(PYTHON_BIN) query.py start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/qldb-ledger-queries/README.md b/qldb-ledger-queries/README.md index 87e4a8f4..c56893c5 100644 --- a/qldb-ledger-queries/README.md +++ b/qldb-ledger-queries/README.md @@ -1,39 +1,52 @@ -# LocalStack Demo: Quantum Ledger Database (QLDB) Queries +# QLDB Ledger Queries -Simple demo application illustrating running queries against a QLDB ledger on your local machine, using LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | QLDB | +| Integrations | AWS CLI | +| Categories | Database | + +## Introduction + +A demo application illustrating running queries against a Quantum Ledger Database (QLDB) ledger locally using LocalStack. The sample creates a QLDB ledger, runs INSERT and SELECT queries, and demonstrates join operations between tables in the ledger journal. ## Prerequisites -* LocalStack -* Docker -* Python -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Run the scenario Python script `query.py` as follows: -``` +```bash make run ``` -The test script creates a new QLDB ledger, and runs a couple of queries to INSERT and SELECT data from the ledger journal. +The script creates a QLDB ledger, inserts test data, runs SELECT queries, and demonstrates join operations across tables. + +You should see output similar to: -You should see some logs from the script, similar to the output below: ``` Scenario 1: create and list tables in ledger ----------- diff --git a/rds-db-queries/Makefile b/rds-db-queries/Makefile index f8d4f17e..bdcd6b22 100644 --- a/rds-db-queries/Makefile +++ b/rds-db-queries/Makefile @@ -7,6 +7,14 @@ PYTHON_BIN ?= $(shell which python3 || which python) usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -16,7 +24,8 @@ run: ## Run the scenario with the DB queries against local RDS source .venv/bin/activate; echo `which python3`; python3 query.py start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/rds-db-queries/README.md b/rds-db-queries/README.md index 461ca78f..978e5e37 100644 --- a/rds-db-queries/README.md +++ b/rds-db-queries/README.md @@ -1,37 +1,52 @@ -# LocalStack Demo: RDS Database Queries +# RDS Database Queries -Simple demo application illustrating running queries against an RDS database. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | RDS | +| Integrations | AWS CLI | +| Categories | Database | + +## Introduction + +A demo application illustrating running queries against an RDS database locally using LocalStack. The sample creates an RDS DB instance, executes SQL INSERT and SELECT queries, and cleans up the instance. ## Prerequisites -* LocalStack -* Docker -* Python -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Run the scenario Python script `query.py` as follows: -``` +```bash make run ``` -You should see some logs from the script, similar to the output below: +The script creates an RDS DB instance, runs queries to insert and retrieve records, and deletes the instance. + +You should see output similar to: + ``` Creating RDS DB instance Run DB queries against RDS instance i1 diff --git a/rds-failover-test/Makefile b/rds-failover-test/Makefile index e71a21e1..185c6ed4 100644 --- a/rds-failover-test/Makefile +++ b/rds-failover-test/Makefile @@ -7,6 +7,14 @@ PYTHON_BIN ?= $(shell which python3 || which python) usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -16,7 +24,8 @@ run: ## Run the scenario with the DB queries against local RDS source .venv/bin/activate; echo `which python3`; python3 main.py start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/rds-failover-test/README.md b/rds-failover-test/README.md index 483fca2d..d51676fa 100644 --- a/rds-failover-test/README.md +++ b/rds-failover-test/README.md @@ -1,42 +1,53 @@ -# LocalStack Demo: RDS Failover Test +# RDS Failover Test -Simple demo application illustrating running a failover test against an RDS database. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | RDS | +| Integrations | AWS CLI | +| Categories | Database; Resilience | + +## Introduction + +A demo application illustrating RDS global cluster failover testing with LocalStack. The sample creates a global RDS cluster with primary and secondary clusters, then triggers a failover to verify that the cluster writer is updated correctly. ## Prerequisites -* LocalStack -* Docker -* Python -* `make` -* [`awslocal`](https://github.com/localstack/awscli-local) +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) +- [Python 3](https://www.python.org/downloads/) + +## Check prerequisites -## Installing +```bash +make check +``` -To install the dependencies: +## Installation ```bash make install ``` -## Starting LocalStack - -Make sure that LocalStack is started: +## Start LocalStack ```bash -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running - -Run the scenario Python script `main.py` as follows: +## Run the application ```bash make run ``` -You should see some logs from the script, similar to the output below: +The script creates a global RDS cluster with primary and secondary instances, triggers a failover, and verifies the cluster writer has been updated. -```bash +You should see output similar to: + +``` Creating global cluster 'global-cluster-1' Creating primary DB cluster 'rds-cluster-1' Creating secondary DB cluster 'rds-cluster-2' diff --git a/reproducible-ml/Makefile b/reproducible-ml/Makefile index 010b9aea..3889943e 100644 --- a/reproducible-ml/Makefile +++ b/reproducible-ml/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION=us-east-1 usage: @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -13,7 +20,8 @@ run: ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/reproducible-ml/README.md b/reproducible-ml/README.md index 3ace5f2a..5bf02b72 100644 --- a/reproducible-ml/README.md +++ b/reproducible-ml/README.md @@ -1,49 +1,68 @@ -# LocalStack Demo: Train, save and evaluate a scikit-learn machine learning model +# Reproducible ML with Lambda and S3 -In this tutorial, we will train a simple machine-learning model that recognizes handwritten digits on an image. -We will use the following services: +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda, S3 | +| Integrations | AWS CLI | +| Categories | ML; Serverless | -* an S3 bucket to host our training data; -* a Lambda function to train and save the model to an S3 bucket; -* a Lambda layer that contains the dependencies for our training code; -* a second Lambda function to download the saved model and perform a prediction with it. +## Introduction + +A demo application illustrating how to train, save, and evaluate a scikit-learn machine learning model using Lambda and S3 with LocalStack. A training Lambda function trains a digit recognition model and stores it in S3, while a second Lambda function downloads the model and runs predictions. + +In this demo application, we will train a simple machine-learning model that recognizes handwritten digits on an image. We will use the following services: + +- An S3 bucket to host the training data; +- A Lambda function to train and save the model to an S3 bucket; +- A Lambda layer that contains the dependencies for the training code; +- A second Lambda function to download the saved model and perform a prediction with it. ## Prerequisites -* LocalStack -* Docker -* `awslocal` CLI +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -The entire workflow is executed by the `run.sh` script. To trigger it, execute: -``` +```bash make run ``` + +The script trains a scikit-learn digit recognition model via a Lambda function, stores it in S3, then invokes a second Lambda to download and evaluate the model. + The model will be first trained by the `ml-train` Lambda function and then uploaded on the S3 bucket. A second Lambda function will download the model and run predictions on a test set of character inputs. -The logs of the Lambda invocation should be visible in the LocalStack container output (with DEBUG=1 enabled): +The logs of the Lambda invocation should be visible in the LocalStack container output (with `DEBUG=1` enabled): -```bash -null +``` >START RequestId: 65dc894d-25e0-168e-dea1-a3e8bfdb563b Version: $LATEST > --> prediction result: [8 8 4 9 0 8 9 8 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 9 6 7 8 9 ... -... > 9 5 4 8 8 4 9 0 8 9 8] -> END RequestId: 6... -``` \ No newline at end of file +> END RequestId: 65dc894d-25e0-168e-dea1-a3e8bfdb563b +``` + +## License + +This code is available under the Apache 2.0 license. diff --git a/route53-dns-failover/Makefile b/route53-dns-failover/Makefile index 8641ba23..30ba9c01 100644 --- a/route53-dns-failover/Makefile +++ b/route53-dns-failover/Makefile @@ -5,6 +5,13 @@ export AWS_DEFAULT_REGION=us-east-1 usage: @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: @which localstack || pip install localstack @which awslocal || pip install awscli-local @@ -13,7 +20,8 @@ run: ./run.sh start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) docker compose up -d stop: @echo diff --git a/route53-dns-failover/README.md b/route53-dns-failover/README.md index 987dd164..6babe334 100644 --- a/route53-dns-failover/README.md +++ b/route53-dns-failover/README.md @@ -1,13 +1,50 @@ -This sample demonstrates the Route53 DNS failover based on health check in LocalStack. +# Route53 DNS Failover -We suggest taking a look at [run.sh](run.sh) script to understand the setup. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Route53, EC2 | +| Integrations | Docker Compose, AWS CLI | +| Categories | Networking; DNS | -To run the demo: +## Introduction +A demo application illustrating Route53 DNS failover based on health checks using LocalStack. The sample configures Route53 health checks and DNS records with failover routing policies, then demonstrates automatic DNS failover when the primary endpoint becomes unhealthy. + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) and Docker Compose +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) + +## Check prerequisites + +```bash +make check ``` -$ LOCALSTACK_AUTH_TOKEN= docker-compose up -d + +## Installation + +```bash +make install ``` +## Start LocalStack + +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -$ make run + +This starts LocalStack and supporting services via Docker Compose. + +## Run the application + +```bash +make run ``` + +The script configures Route53 health checks and DNS failover routing, then demonstrates failover behavior. + +## License + +This code is available under the Apache 2.0 license. diff --git a/sagemaker-inference/Makefile b/sagemaker-inference/Makefile new file mode 100644 index 00000000..a8ba5993 --- /dev/null +++ b/sagemaker-inference/Makefile @@ -0,0 +1,40 @@ +export AWS_ACCESS_KEY_ID ?= test +export AWS_SECRET_ACCESS_KEY ?= test +export AWS_DEFAULT_REGION ?= us-east-1 +SHELL := /bin/bash + +usage: ## Show this help + @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' + +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @echo "All required prerequisites are available." + +install: ## Install Python dependencies + @pip install -r requirements.txt + +start: ## Start LocalStack + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + localstack start -d + +stop: ## Stop LocalStack + @echo + localstack stop + +ready: ## Wait until LocalStack is ready + @echo Waiting on the LocalStack container... + @localstack wait -t 30 && echo LocalStack is ready to use! || (echo Gave up waiting on LocalStack, exiting. && exit 1) + +logs: ## Retrieve LocalStack logs + @localstack logs > logs.txt + +run: ## Run the SageMaker inference example + @python3 main.py + +test-ci: ## Run CI tests + make check start install ready run; return_code=`echo $$?`;\ + make logs; make stop; exit $$return_code; + +.PHONY: usage check install start stop ready logs run test-ci diff --git a/sagemaker-inference/ReadMe.md b/sagemaker-inference/ReadMe.md index fcaffc3d..7b4fcf0f 100644 --- a/sagemaker-inference/ReadMe.md +++ b/sagemaker-inference/ReadMe.md @@ -1,53 +1,63 @@ # SageMaker Model Inference -This is a small example about how you can use LocalStack to host your PyTorch ML models. It does the following: +| Key | Value | +| ------------ | ----------------------------------- | +| Services | SageMaker, S3 | +| Integrations | AWS SDK (boto3) | +| Categories | ML; Inference | -* Create MNIST model in SageMaker -* Create a SageMaker Endpoint for accessing the model -* Invoke the endpoint - * directly on the container - * via boto +## Introduction -## Requirements +A demo application illustrating how to host PyTorch ML models with SageMaker using LocalStack. The sample creates a SageMaker endpoint for an MNIST digit recognition model and demonstrates invocations both directly on the container and via the boto3 SDK. -* Python 3.8+ - * boto3 - * numpy - * mypy -* LocalStack -* Docker +### Obtain the Deep Learning Image -## How To - -### Obtain Deep Learning image - -Before using this example you should set up your Docker Client to pull the AWS Deep Learning images ([more info here](https://github.com/aws/deep-learning-containers/blob/master/available_images.md)): +Before running this example, set up your Docker client to pull AWS Deep Learning images ([more info](https://github.com/aws/deep-learning-containers/blob/master/available_images.md)): ```bash aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com ``` -Because the images tend to be heavy (multiple GB), you might want to `docker pull` them beforehand: +Because the images tend to be heavy (multiple GB), pull them beforehand: ```bash docker pull 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.5.0-cpu-py3 ``` -### Test the application +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [Python 3.8+](https://www.python.org/downloads/) + +## Check prerequisites + +```bash +make check +``` + +## Installation + +```bash +make install +``` -Afterwards you can start localstack: +## Start LocalStack ```bash -localstack start +export LOCALSTACK_AUTH_TOKEN= +make start ``` -And execute the example with: +## Run the application ```bash -python main.py +make run ``` -You should see an output like this for each of the runs: +The script creates an S3 bucket, uploads model data, creates a SageMaker endpoint, and invokes it to predict digit classes. You should see output similar to: + ``` Creating bucket... Uploading model data to bucket... @@ -64,4 +74,8 @@ Invoking endpoint directly... Predicted digits: [2, 6] ``` -To try out the serverless run you can remove the comment in the `main.py` file and run the example again. \ No newline at end of file +To try out the serverless run, remove the comment in `main.py` and run the example again. + +## License + +This code is available under the Apache 2.0 license. diff --git a/serverless-lambda-layers/Makefile b/serverless-lambda-layers/Makefile index 886bdc49..905db661 100644 --- a/serverless-lambda-layers/Makefile +++ b/serverless-lambda-layers/Makefile @@ -5,6 +5,14 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @npm install @which serverless || npm install -g serverless @@ -20,7 +28,8 @@ run: ## Deploy the app locally and run a Lambda test invocation awslocal lambda invoke --function-name test-layers-local-hello /tmp/out.tmp start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/serverless-lambda-layers/README.md b/serverless-lambda-layers/README.md index 52efb25a..86335494 100644 --- a/serverless-lambda-layers/README.md +++ b/serverless-lambda-layers/README.md @@ -1,41 +1,60 @@ -# LocalStack Demo: Lambda Layers +# Serverless Lambda Layers -Simple demo application illustrating Lambda layers using LocalStack, deployed via the Serverless framework. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Lambda | +| Integrations | Serverless Framework | +| Categories | Serverless | + +## Introduction + +A demo application illustrating Lambda layers using LocalStack, deployed via the Serverless framework. Lambda layers allow you to package shared code and dependencies separately from your function code, enabling reuse across multiple functions. ## Prerequisites -* LocalStack -* Docker -* Node.js / `npm` -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [Node.js](https://nodejs.org/en/download/) with `npm` +- [Serverless Framework](https://www.serverless.com/framework/docs/getting-started) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -Deploy the app locally and run a Lambda test invocation: -``` +## Run the application + +```bash make run ``` -You should see a success output in the terminal: +The script deploys the Lambda function with its layer via the Serverless framework and invokes it to verify the layer is loaded correctly. + +You should see a success output: + ``` { "StatusCode": 200 } ``` -... and your LocalStack container should contain output similar to this: +Check the LocalStack container logs for the Lambda output: + ``` >START RequestId: ba4efc87-7bf9-1705-9f45-8e84ba8eb071 Version: $LATEST > 2019-10-23T14:25:12.709Z ba4efc87-7bf9-1705-9f45-8e84ba8eb071 INFO This text should be printed in the Lambda diff --git a/serverless-websockets/Makefile b/serverless-websockets/Makefile index b2b4d4a7..342d4e91 100644 --- a/serverless-websockets/Makefile +++ b/serverless-websockets/Makefile @@ -6,6 +6,15 @@ export AWS_DEFAULT_REGION = us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v node > /dev/null 2>&1 || { echo "Node.js is not installed. Please install Node.js and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @npm install @which serverless || npm install -g serverless @@ -22,7 +31,8 @@ run: ## Deploy and run the app locally start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/serverless-websockets/README.md b/serverless-websockets/README.md index 958c9b7d..95ca944e 100644 --- a/serverless-websockets/README.md +++ b/serverless-websockets/README.md @@ -1,36 +1,52 @@ -# LocalStack Demo: Websockets via API Gateway V2 +# Serverless WebSockets -Simple demo application illustrating API Gateway V2 websocket APIs using LocalStack, deployed via the Serverless framework. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | API Gateway, Lambda | +| Integrations | Serverless Framework | +| Categories | Serverless; WebSockets | + +## Introduction + +A demo application illustrating API Gateway V2 WebSocket APIs using LocalStack, deployed via the Serverless framework. The sample deploys a Lambda function connected to a WebSocket API and demonstrates bidirectional message passing over the WebSocket connection. ## Prerequisites -* LocalStack -* Docker -* Node.js / `npm` -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [Node.js](https://nodejs.org/en/download/) with `npm` +- [Serverless Framework](https://www.serverless.com/framework/docs/getting-started) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Starting LocalStack +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -## Running +## Run the application -Deploy the app locally and send a test message to the created Websockets API. -``` +```bash make run ``` -This should trigger a Lambda invocation which simply writes the invocation payload back to the websocket. You should see a successful output in the terminal: +The script deploys the WebSocket API via Serverless, connects a WebSocket client, sends a test message, and verifies it is echoed back by the Lambda handler. + +You should see output similar to: + ``` ... Serverless: Stack create finished... diff --git a/stepfunctions-lambda/Makefile b/stepfunctions-lambda/Makefile index ca961551..0a297f60 100644 --- a/stepfunctions-lambda/Makefile +++ b/stepfunctions-lambda/Makefile @@ -5,6 +5,14 @@ export AWS_DEFAULT_REGION=us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which awslocal || pip install awscli-local @@ -49,6 +57,7 @@ create-lambdas: ## Create the Lambdas rm package.zip lambda_function.py start: + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) localstack start -d stop: diff --git a/stepfunctions-lambda/README.md b/stepfunctions-lambda/README.md index 9e2fb5cd..fdd573e4 100644 --- a/stepfunctions-lambda/README.md +++ b/stepfunctions-lambda/README.md @@ -1,43 +1,55 @@ -# Creating a Lambda function with a function URL +# Step Functions with Lambda -In this example, we will demonstrate how to create an AWS Step Function with Lambda functions in LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Step Functions, Lambda | +| Integrations | AWS CLI | +| Categories | Serverless; Orchestration | -## Prerequisites +## Introduction + +A demo application illustrating how to orchestrate Lambda functions using AWS Step Functions with LocalStack. The sample creates multiple Lambda functions and a Step Functions state machine that coordinates their execution. -* LocalStack -* Docker -* `awslocal` CLI +## Prerequisites -## Starting up +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -Start LocalStack via: +## Check prerequisites -```sh -localstack start -d +```bash +make check ``` -Run the following command to create the Lambda functions: +## Installation -```sh -make create-lambdas +```bash +make install ``` -## Setting up and running Step Function +## Start LocalStack -Create the Step Function: +```bash +export LOCALSTACK_AUTH_TOKEN= +make start +``` -```sh -awslocal stepfunctions create-state-machine --name step-demo \ - --definition "$(cat step-definition.json)" \ - --role-arn arn:aws:iam::000000000000:role/step-function-lambda +## Deploy the Application + +```bash +make deploy ``` -Start the execution: +## Run the application -```sh -awslocal stepfunctions start-execution \ - --state-machine-arn arn:aws:states:us-east-1:000000000000:stateMachine:step-demo \ - --input '{"adam": "LocalStack", "cole": "Stack"}' +```bash +make run ``` -This creates and invokes the flow between the three Lambda functions we created using LocalStack earlier. +The script creates Lambda functions and a Step Functions state machine, then starts an execution that flows through the Lambda orchestration. + +## License + +This code is available under the Apache 2.0 license. diff --git a/terraform-resources/Makefile b/terraform-resources/Makefile index 172adab1..9d7e97d8 100644 --- a/terraform-resources/Makefile +++ b/terraform-resources/Makefile @@ -5,6 +5,14 @@ export PATH := $(PATH):. usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v terraform > /dev/null 2>&1 || { echo "Terraform is not installed. Please install Terraform and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies @which terraform || (\ echo 'Terraform was not found, installing locally' && \ @@ -26,7 +34,8 @@ uninstall: @rm -f terraform start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/terraform-resources/README.md b/terraform-resources/README.md index 2bde129b..d956e852 100644 --- a/terraform-resources/README.md +++ b/terraform-resources/README.md @@ -1,44 +1,50 @@ -# LocalStack Demo: Create Local AWS Resources via Terraform +# Terraform Resources -Simple demo project deploying various AWS resources to LocalStack via Terraform. +| Key | Value | +| ------------ | ---------------------------------------------------------------- | +| Services | S3, Lambda, API Gateway, RDS, ElastiCache, IAM | +| Integrations | Terraform | +| Categories | IaC | + +## Introduction + +A demo application deploying various AWS resources to LocalStack via Terraform. The sample creates a range of AWS resources including S3 buckets, Lambda functions, API Gateway endpoints, RDS parameter groups, and ElastiCache clusters — all using `tflocal` to redirect Terraform to LocalStack. ## Prerequisites -* LocalStack -* Docker -* Terraform -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [Terraform](https://developer.hashicorp.com/terraform/downloads) -## Installing +## Check prerequisites -Install the dependencies using this command: -``` -make install +```bash +make check ``` -## Starting LocalStack +## Installation -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +make install ``` -## Running +## Start LocalStack -Create the resources via Terraform locally in LocalStack: -``` +```bash +export LOCALSTACK_AUTH_TOKEN= make start ``` -You may need to confirm the creation by entering "yes". You should then see log output similar to the one below: +## Deploy the Application + +```bash +make run ``` -... - Do you want to perform these actions? - Terraform will perform the actions described above. - Only 'yes' will be accepted to approve. - Enter a value: yes +This initializes Terraform (if not already done) and runs `terraform apply --auto-approve`. You may need to confirm the creation by entering `yes`. You should see output similar to: +``` Plan: 10 to add, 0 to change, 0 to destroy. aws_iam_role.invocation_role: Creating... aws_api_gateway_rest_api.demo: Creating... @@ -47,21 +53,7 @@ aws_db_parameter_group.default: Creating... aws_elasticache_cluster.my-redis: Creating... aws_s3_bucket.test-bucket: Creating... aws_api_gateway_rest_api.demo: Creation complete after 1s [id=iq0njx2s0a] -aws_iam_role.invocation_role: Creation complete after 1s [id=api_gateway_auth_invocation] -aws_iam_role.lambda: Creation complete after 1s [id=demo-lambda] -aws_iam_role_policy.invocation_policy: Creating... -aws_lambda_function.authorizer: Creating... -aws_iam_role_policy.invocation_policy: Creation complete after 0s [id=api_gateway_auth_invocation:default] -aws_s3_bucket.test-bucket: Creation complete after 2s [id=my-bucket] -aws_s3_bucket_acl.test-bucket-acl: Creating... -aws_s3_bucket_acl.test-bucket-acl: Creation complete after 0s [id=my-bucket,private] -aws_db_parameter_group.default: Creation complete after 2s [id=rds-pg] -aws_lambda_function.authorizer: Creation complete after 7s [id=api_gateway_authorizer] -aws_api_gateway_authorizer.demo: Creating... -aws_api_gateway_authorizer.demo: Creation complete after 0s [id=9a2570] -aws_elasticache_cluster.my-redis: Still creating... [10s elapsed] -aws_elasticache_cluster.my-redis: Still creating... [20s elapsed] -aws_elasticache_cluster.my-redis: Still creating... [30s elapsed] +... aws_elasticache_cluster.my-redis: Creation complete after 32s [id=my-redis-cluster] Apply complete! Resources: 10 added, 0 changed, 0 destroyed. diff --git a/testcontainers-java-sample/README.md b/testcontainers-java-sample/README.md index 7b5ac140..0e029d99 100644 --- a/testcontainers-java-sample/README.md +++ b/testcontainers-java-sample/README.md @@ -1,14 +1,30 @@ -# Example using RDS with Localstack Testcontainers +# RDS with LocalStack Testcontainers -Testcontainers need a special setup to use services like RDS, which may use any port to expose the database. -The sample explains how the mapping works, and how you need to configure Testcontainers in order to connect to the RDS instance from your test. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | RDS | +| Integrations | Testcontainers, Java | +| Categories | Database; Testing | + +## Introduction + +A demo application illustrating how to use LocalStack Testcontainers with RDS in Java. Testcontainers requires a special setup for RDS because the service may expose the database on any port. The sample demonstrates the port mapping configuration needed to connect to an RDS PostgreSQL instance from your test code. + +## Prerequisites + +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [Java](https://openjdk.org/) and [Maven](https://maven.apache.org/) ## Run Example -* Import the project (e.g. in IntelliJ), -* configure your LOCALSTACK_AUTH_TOKEN as environment variable, -* and then run the test `TestRDS`. -It will create a LocalStack Testcontainer and a postgres database instance using RDSClient. -The database will then be filled with some data, and queried afterwards. +- Import the project (e.g. in IntelliJ), +- Configure your LOCALSTACK_AUTH_TOKEN as environment variable, +- Run the test `TestRDS` in your IDE. + +It will create a LocalStack Testcontainer and a PostgreSQL database instance using `RDSClient`. +The database will then be filled with some data, and queried afterwards. +## License +This code is available under the Apache 2.0 license. diff --git a/transfer-ftp-s3/Makefile b/transfer-ftp-s3/Makefile index ddcd5f81..54788a5e 100644 --- a/transfer-ftp-s3/Makefile +++ b/transfer-ftp-s3/Makefile @@ -5,6 +5,14 @@ export AWS_DEFAULT_REGION=us-east-1 usage: ## Show this help @fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//' +check: ## Check if all required prerequisites are installed + @command -v docker > /dev/null 2>&1 || { echo "Docker is not installed. Please install Docker and try again."; exit 1; } + @command -v python3 > /dev/null 2>&1 || { echo "Python 3 is not installed. Please install Python 3 and try again."; exit 1; } + @command -v localstack > /dev/null 2>&1 || { echo "LocalStack CLI is not installed. Please install it and try again."; exit 1; } + @command -v awslocal > /dev/null 2>&1 || { echo "awslocal is not installed. Please run: pip install awscli-local"; exit 1; } + @echo "All required prerequisites are available." + + install: ## Install dependencies # @npm install @which localstack || pip install localstack @@ -16,7 +24,8 @@ run: ## Deploy and run the sample locally (. .venv/bin/activate; python test.py) start: - localstack start -d + @test -n "${LOCALSTACK_AUTH_TOKEN}" || (echo "LOCALSTACK_AUTH_TOKEN is not set. Find your token at https://app.localstack.cloud/workspace/auth-token"; exit 1) + DEBUG=1 LOCALSTACK_AUTH_TOKEN=$(LOCALSTACK_AUTH_TOKEN) localstack start -d stop: @echo diff --git a/transfer-ftp-s3/README.md b/transfer-ftp-s3/README.md index ca5b821c..0c4e6613 100644 --- a/transfer-ftp-s3/README.md +++ b/transfer-ftp-s3/README.md @@ -1,33 +1,51 @@ -# LocalStack Demo: FTP Upload to S3 Bucket via AWS Transfer API +# FTP Upload to S3 via AWS Transfer API -Simple demo application illustrating the use of AWS Transfer API in LocalStack. +| Key | Value | +| ------------ | ----------------------------------- | +| Services | Transfer, S3 | +| Integrations | AWS CLI | +| Categories | File Transfer; Storage | + +## Introduction + +A demo application illustrating the use of the AWS Transfer API with LocalStack. The sample creates a local FTP server via the Transfer API, uploads files via FTP, and downloads the uploaded files from the target S3 bucket. ## Prerequisites -* LocalStack -* Docker -* `make` +- A valid [LocalStack for AWS license](https://localstack.cloud/pricing). Your license provides a [`LOCALSTACK_AUTH_TOKEN`](https://docs.localstack.cloud/getting-started/auth-token/) to activate LocalStack. +- [Docker](https://docs.docker.com/get-docker/) +- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli) +- [`awslocal` CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) -## Installing +## Check prerequisites -To install the dependencies: +```bash +make check ``` + +## Installation + +```bash make install ``` -## Running +## Start LocalStack -Make sure that LocalStack is started: -``` -LOCALSTACK_AUTH_TOKEN=... DEBUG=1 localstack start +```bash +export LOCALSTACK_AUTH_TOKEN= +make start ``` -The following command runs the test application locally, creates an FTP server via AWS Transfer API locally, uploads two files via FTP, and downloads the files from the target S3 bucket: -``` +## Run the application + +```bash make run ``` -You should then see a couple of log messages in the terminal: +The script creates an FTP server via the AWS Transfer API, uploads two files (to root and a sub-directory), then downloads and verifies them from the target S3 bucket. + +You should then see log messages similar to: + ``` Running Test: Creating FTP server and uploading files to S3 via Transfer API Creating FTP server in AWS Transfer API