Skip to content

Commit f8e365e

Browse files
committed
feat: add initial operations
Initial test framework to run AWS Durable Functions locally in a unit test environment. Includes validation for: - step - wait - run_in_child_context - create_callback - wait_for_callback - wait_for_condition - parallel - map
1 parent 461678a commit f8e365e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+11809
-9
lines changed

.github/workflows/ci.yml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
2+
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
3+
4+
name: Python package
5+
6+
on:
7+
push:
8+
9+
pull_request:
10+
branches: [ main ]
11+
12+
jobs:
13+
build:
14+
15+
runs-on: ubuntu-latest
16+
strategy:
17+
fail-fast: false
18+
matrix:
19+
python-version: ["3.11", "3.13"]
20+
21+
steps:
22+
- uses: actions/checkout@v5
23+
- name: Set up Python ${{ matrix.python-version }}
24+
uses: actions/setup-python@v6
25+
with:
26+
python-version: ${{ matrix.python-version }}
27+
- name: Install Hatch
28+
run: |
29+
python -m pip install --upgrade hatch
30+
- name: static analysis
31+
run: hatch fmt --check
32+
- name: type checking
33+
run: hatch run types:check
34+
- name: Run tests + coverage
35+
run: hatch run test:cov
36+
- name: Build distribution
37+
run: hatch build

.gitignore

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
*~
2+
*#
3+
*.swp
4+
*.iml
5+
*.DS_Store
6+
7+
__pycache__/
8+
*.py[cod]
9+
*$py.class
10+
*.egg-info/
11+
12+
/.coverage
13+
/.coverage.*
14+
/.cache
15+
/.pytest_cache
16+
/.mypy_cache
17+
18+
/doc/_apidoc/
19+
/build
20+
21+
.venv
22+
.venv/
23+
24+
.attach_*
25+
26+
dist/

CONTRIBUTING.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,119 @@ documentation, we greatly value feedback and contributions from our community.
66
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
77
information to effectively respond to your bug report or contribution.
88

9+
## Dependencies
10+
Install [hatch](https://hatch.pypa.io/dev/install/).
11+
12+
## Developer workflow
13+
These are all the checks you would typically do as you prepare a PR:
14+
```
15+
# just test
16+
hatch test
17+
18+
# coverage
19+
hatch run test:cov
20+
21+
# type checks
22+
hatch run types:check
23+
24+
# static analysis
25+
hatch fmt
26+
```
27+
28+
## Set up your IDE
29+
Point your IDE at the hatch virtual environment to have it recognize dependencies
30+
and imports.
31+
32+
You can find the path to the hatch Python interpreter like this:
33+
```
34+
echo "$(hatch env find)/bin/python"
35+
```
36+
37+
### VS Code
38+
If you're using VS Code, "Python: Select Interpreter" and use the hatch venv Python interpreter
39+
as found with the `hatch env find` command.
40+
41+
Hatch uses Ruff for static analysis.
42+
43+
You might want to install the [Ruff extension for VS Code](https://github.com/astral-sh/ruff-vscode)
44+
to have your IDE interactively warn of the same linting and formatting rules.
45+
46+
These `settings.json` settings are useful:
47+
```
48+
{
49+
"[python]": {
50+
"editor.formatOnSave": true,
51+
"editor.codeActionsOnSave": {
52+
"source.fixAll": "explicit",
53+
"source.organizeImports": "explicit"
54+
},
55+
"editor.defaultFormatter": "charliermarsh.ruff"
56+
},
57+
"ruff.nativeServer": "on"
58+
}
59+
```
60+
61+
## Testing
62+
### How to run tests
63+
To run all tests:
64+
```
65+
hatch test
66+
```
67+
68+
To run a single test file:
69+
```
70+
hatch test tests/path_to_test_module.py
71+
```
72+
73+
To run a specific test in a module:
74+
```
75+
hatch test tests/path_to_test_module.py::test_mytestmethod
76+
```
77+
78+
To run a single test, or a subset of tests:
79+
```
80+
$ hatch test -k TEST_PATTERN
81+
```
82+
83+
This will run tests which contain names that match the given string expression (case-insensitive),
84+
which can include Python operators that use filenames, class names and function names as variables.
85+
86+
### Debug
87+
To debug failing tests:
88+
89+
```
90+
$ hatch test --pdb
91+
```
92+
93+
This will drop you into the Python debugger on the failed test.
94+
95+
### Writing tests
96+
Place test files in the `tests/` directory, using file names that end with `_test`.
97+
98+
Mimic the package structure in the src/aws_durable_functions_sdk_python directory.
99+
Name your module so that src/mypackage/mymodule.py has a dedicated unit test file
100+
tests/mypackage/mymodule_test.py
101+
102+
## Coverage
103+
```
104+
hatch run test:cov
105+
```
106+
107+
## Linting and type checks
108+
Type checking:
109+
```
110+
hatch run types:check
111+
```
112+
113+
Static analysis (with auto-fix of known issues):
114+
```
115+
hatch fmt
116+
```
117+
118+
To do static analysis without auto-fixes:
119+
```
120+
hatch fmt --check
121+
```
9122

10123
## Reporting Bugs/Feature Requests
11124

README.md

Lines changed: 171 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,179 @@
1-
## My Project
1+
# aws-durable-functions-sdk-python
22

3-
TODO: Fill this README out!
3+
[![PyPI - Version](https://img.shields.io/pypi/v/aws-durable-functions-sdk-python.svg)](https://pypi.org/project/aws-durable-functions-sdk-python)
4+
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/aws-durable-functions-sdk-python.svg)](https://pypi.org/project/aws-durable-functions-sdk-python)
45

5-
Be sure to:
6+
-----
67

7-
* Change the title in this README
8-
* Edit your repository description on GitHub
8+
## Table of Contents
99

10-
## Security
10+
- [Installation](#installation)
11+
- [Quick Start](#quick-start)
12+
- [Architecture](#architecture)
13+
- [Developer Guide](#developers)
14+
- [License](#license)
1115

12-
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
16+
## Installation
1317

14-
## License
18+
```console
19+
pip install aws-durable-functions-sdk-python-testing
20+
```
21+
22+
## Overview
23+
24+
Use the Durable Functions Python Testing Framework to test your Python Durable Functions locally.
25+
26+
The test framework contains a local runner, so you can run and test your Durable Function locally
27+
before you deploy it.
28+
29+
## Quick Start
30+
31+
### A Durable Function under test
32+
33+
```python
34+
from durable_executions_python_language_sdk.context import (
35+
DurableContext,
36+
durable_step,
37+
durable_with_child_context,
38+
)
39+
from durable_executions_python_language_sdk.execution import durable_handler
40+
41+
@durable_step
42+
def one(a: int, b: int) -> str:
43+
return f"{a} {b}"
44+
45+
@durable_step
46+
def two_1(a: int, b: int) -> str:
47+
return f"{a} {b}"
48+
49+
@durable_step
50+
def two_2(a: int, b: int) -> str:
51+
return f"{b} {a}"
52+
53+
@durable_with_child_context
54+
def two(ctx: DurableContext, a: int, b: int) -> str:
55+
two_1_result: str = ctx.step(two_1(a, b))
56+
two_2_result: str = ctx.step(two_2(a, b))
57+
return f"{two_1_result} {two_2_result}"
58+
59+
@durable_step
60+
def three(a: int, b: int) -> str:
61+
return f"{a} {b}"
62+
63+
@durable_handler
64+
def function_under_test(event: Any, context: DurableContext) -> list[str]:
65+
results: list[str] = []
66+
67+
result_one: str = context.step(one(1, 2))
68+
results.append(result_one)
69+
70+
context.wait(seconds=1)
71+
72+
result_two: str = context.run_in_child_context(two(3, 4))
73+
results.append(result_two)
74+
75+
result_three: str = context.step(three(5, 6))
76+
results.append(result_three)
77+
78+
return results
79+
```
80+
81+
### Your test code
82+
83+
```python
84+
from aws_durable_functions_sdk_python.execution import InvocationStatus
85+
from aws_durable_functions_sdk_python_testing.runner import (
86+
ContextOperation,
87+
DurableFunctionTestResult,
88+
DurableFunctionTestRunner,
89+
StepOperation,
90+
)
91+
92+
def test_my_durable_functions():
93+
with DurableFunctionTestRunner(handler=function_under_test) as runner:
94+
result: DurableFunctionTestResult = runner.run(input="input str", timeout=10)
1595

16-
This project is licensed under the Apache-2.0 License.
96+
assert result.status is InvocationStatus.SUCCEEDED
97+
assert result.result == '["1 2", "3 4 4 3", "5 6"]'
98+
99+
one_result: StepOperation = result.get_step("one")
100+
assert one_result.result == '"1 2"'
101+
102+
two_result: ContextOperation = result.get_context("two")
103+
assert two_result.result == '"3 4 4 3"'
104+
105+
three_result: StepOperation = result.get_step("three")
106+
assert three_result.result == '"5 6"'
107+
```
108+
## Architecture
109+
![Durable Functions Python Test Framework Architecture](/assets/dar-python-test-framework-architecture.svg)
110+
111+
## Event Flow
112+
![Event Flow Sequence Diagram](/assets/dar-python-test-framework-event-flow.svg)
113+
114+
1. **DurableTestRunner** starts execution via **Executor**
115+
2. **Executor** creates **Execution** and schedules initial invocation
116+
3. During execution, checkpoints are processed by **CheckpointProcessor**
117+
4. **Individual Processors** transform operation updates and may trigger events
118+
5. **ExecutionNotifier** broadcasts events to **Executor** (observer)
119+
6. **Executor** updates **Execution** state based on events
120+
7. **Execution** completion triggers final event notifications
121+
8. **DurableTestRunner** run() blocks until it receives completion event, and then returns `DurableFunctionTestResult`.
122+
123+
## Major Components
124+
125+
### Core Execution Flow
126+
- **DurableTestRunner** - Main entry point that orchestrates test execution
127+
- **Executor** - Manages execution lifecycle. Mutates Execution.
128+
- **Execution** - Represents the state and operations of a single durable execution
129+
130+
### Service Client Integration
131+
- **InMemoryServiceClient** - Replaces AWS Lambda service client for local testing. Injected into SDK via `DurableExecutionInvocationInputWithClient`
132+
133+
### Checkpoint Processing Pipeline
134+
- **CheckpointProcessor** - Orchestrates operation transformations and validation
135+
- **Individual Validators** - Validate operation updates and state transitions
136+
- **Individual Processors** - Transform operation updates into operations (step, wait, callback, context, execution)
137+
138+
### Execution status changes (Observer Pattern)
139+
- **ExecutionNotifier** - Notifies observers of execution events
140+
- **ExecutionObserver** - Interface for receiving execution lifecycle events
141+
- **Executor** implements `ExecutionObserver` to handle completion events
142+
143+
## Component Relationships
144+
145+
### 1. DurableTestRunner → Executor → Execution
146+
- **DurableTestRunner** serves as the main API entry point and sets up all components
147+
- **Executor** manages the execution lifecycle, handling invocations and state transitions
148+
- **Execution** maintains the state of operations and completion status
149+
150+
### 2. Service Client Injection
151+
- **DurableTestRunner** creates **InMemoryServiceClient** with **CheckpointProcessor**
152+
- **InProcessInvoker** injects the service client into SDK via `DurableExecutionInvocationInputWithClient`
153+
- When durable functions call checkpoint operations, they're intercepted by **InMemoryServiceClient**
154+
- **InMemoryServiceClient** delegates to **CheckpointProcessor** for local processing
155+
156+
### 3. CheckpointProcessor → Individual Validators → Individual Processors
157+
- **CheckpointProcessor** orchestrates the checkpoint processing pipeline
158+
- **Individual Validators** (CheckpointValidator, TransitionsValidator, and operation-specific validators) ensure operation updates are valid
159+
- **Individual Processors** (StepProcessor, WaitProcessor, etc.) transform `OperationUpdate` into `Operation`
160+
161+
### 4. Observer Pattern Flow
162+
The observer pattern enables loose coupling between checkpoint processing and execution management:
163+
164+
1. **CheckpointProcessor** processes operation updates
165+
2. **Individual Processors** detect state changes (completion, failures, timer scheduling)
166+
3. **ExecutionNotifier** broadcasts events to registered observers
167+
4. **Executor** (as ExecutionObserver) receives notifications and updates **Execution** state
168+
5. **Execution** complete_* methods finalize the execution state
169+
170+
171+
## Developers
172+
Please see [CONTRIBUTING.md](CONTRIBUTING.md). It contains the testing guide, sample commands and instructions
173+
for how to contribute to this package.
174+
175+
tldr; use `hatch` and it will manage virtual envs and dependencies for you, so you don't have to do it manually.
176+
177+
## License
17178

179+
This project is licensed under the [Apache-2.0 License](LICENSE).

assets/dar-python-test-framework-architecture.svg

Lines changed: 1 addition & 0 deletions
Loading

0 commit comments

Comments
 (0)