|
| 1 | +# temporal-parseable |
| 2 | + |
| 3 | +Temporal plugin that ships workflow and activity execution events to [Parseable](https://parseable.com) as OpenTelemetry structured logs and traces. |
| 4 | + |
| 5 | +``` |
| 6 | +┌──────────────┐ OTLP/HTTP ┌──────────────┐ |
| 7 | +│ Temporal │ ──────────────── │ Parseable │ |
| 8 | +│ Worker │ logs + traces │ │ |
| 9 | +│ + Plugin │ │ temporal-* │ |
| 10 | +└──────────────┘ └──────────────┘ |
| 11 | +``` |
| 12 | + |
| 13 | +Two streams in Parseable: |
| 14 | + |
| 15 | +- **`temporal-logs`** — flat queryable records: workflow/activity start, complete, fail, retry, duration, signals, queries, updates, child workflows, continue-as-new, and custom domain events |
| 16 | +- **`temporal-traces`** — OTel waterfall traces across workflow and activity boundaries |
| 17 | + |
| 18 | +## Installation |
| 19 | + |
| 20 | +```bash |
| 21 | +pip install temporal-parseable |
| 22 | +``` |
| 23 | + |
| 24 | +## Quick start |
| 25 | + |
| 26 | +```python |
| 27 | +from temporalio.client import Client |
| 28 | +from temporalio.worker import Worker |
| 29 | +from temporal_parseable import ParseablePlugin, ParseableConfig |
| 30 | + |
| 31 | +config = ParseableConfig( |
| 32 | + service_name="my-worker", |
| 33 | + endpoint="https://parseable.example.com", |
| 34 | + username="admin", |
| 35 | + password="secret", |
| 36 | +) |
| 37 | +plugin = ParseablePlugin(config) |
| 38 | + |
| 39 | +client = await Client.connect("localhost:7233", plugins=[plugin]) |
| 40 | + |
| 41 | +async with Worker( |
| 42 | + client, |
| 43 | + task_queue="my-queue", |
| 44 | + workflows=[MyWorkflow], |
| 45 | + activities=[my_activity], |
| 46 | + plugins=[plugin], |
| 47 | +): |
| 48 | + await asyncio.Event().wait() |
| 49 | +``` |
| 50 | + |
| 51 | +## Configuration |
| 52 | + |
| 53 | +All settings fall back to environment variables with the `PARSEABLE_` prefix: |
| 54 | + |
| 55 | +| Argument | Environment variable | Default | |
| 56 | +|---|---|---| |
| 57 | +| `endpoint` | `PARSEABLE_URL` | `http://localhost:8000` | |
| 58 | +| `username` | `PARSEABLE_USERNAME` | `admin` | |
| 59 | +| `password` | `PARSEABLE_PASSWORD` | `admin` | |
| 60 | +| `service_name` | `PARSEABLE_SERVICE_NAME` | `temporal-worker` | |
| 61 | +| `logs.stream` | `PARSEABLE_LOGS_STREAM` | `temporal-logs` | |
| 62 | +| `traces.stream` | `PARSEABLE_TRACES_STREAM` | `temporal-traces` | |
| 63 | + |
| 64 | +Pass `logs=None` or `traces=None` to disable either pipeline. |
| 65 | + |
| 66 | +## Custom domain events |
| 67 | + |
| 68 | +Emit replay-safe domain events from inside workflow code: |
| 69 | + |
| 70 | +```python |
| 71 | +from temporal_parseable.workflow import workflow_event |
| 72 | + |
| 73 | +@workflow.defn |
| 74 | +class AgentWorkflow: |
| 75 | + @workflow.run |
| 76 | + async def run(self, input: AgentInput) -> AgentResult: |
| 77 | + workflow_event("agent.started", {"user_id": input.user_id}) |
| 78 | + |
| 79 | + plan = await workflow.execute_activity(plan_activity, input) |
| 80 | + workflow_event("agent.plan.chosen", {"steps": len(plan.steps)}) |
| 81 | + |
| 82 | + for step in plan.steps: |
| 83 | + workflow_event("agent.step.start", {"tool": step.tool}) |
| 84 | + await workflow.execute_activity(run_step, step) |
| 85 | + |
| 86 | + return result |
| 87 | +``` |
| 88 | + |
| 89 | +Each call emits a record with `type: "user_event"`, `event_name`, and `event_data`. Records are replay-safe — never duplicated during Temporal history replay. |
| 90 | + |
| 91 | +## Log schema |
| 92 | + |
| 93 | +| Field | Type | Notes | |
| 94 | +|---|---|---| |
| 95 | +| `type` | `activity` \| `workflow` \| `user_event` \| `signal` \| `query` \| `update` \| `child_workflow` \| `continue_as_new` | discriminator | |
| 96 | +| `status` | `started` \| `completed` \| `failed` | not on `user_event` | |
| 97 | +| `service_name` | string | from plugin config | |
| 98 | +| `timestamp` | ISO 8601 | event time | |
| 99 | +| `workflow_id` | string | | |
| 100 | +| `run_id` | string | | |
| 101 | +| `workflow_name` | string | | |
| 102 | +| `activity_name` | string | activity records only | |
| 103 | +| `activity_id` | string | activity records only | |
| 104 | +| `attempt` | int | activity records only (1-based) | |
| 105 | +| `duration_ms` | float | on completion/fail | |
| 106 | +| `error` | string | on fail | |
| 107 | +| `direction` | `inbound` \| `outbound` | message records | |
| 108 | +| `message_name` | string | signal/query/update name | |
| 109 | +| `target_workflow_id` | string | outbound signals/child workflows | |
| 110 | +| `event_name` | string | user events only | |
| 111 | +| `event_data` | object | user events only | |
| 112 | + |
| 113 | +## Replay safety |
| 114 | + |
| 115 | +All workflow-side emission is replay-safe. The plugin guards every emit with `workflow.unsafe.is_replaying()`, so records are never duplicated when Temporal replays workflow history (worker crash, cache eviction, manual replay). |
| 116 | + |
| 117 | +## Example queries in Parseable |
| 118 | + |
| 119 | +```sql |
| 120 | +-- Recent workflow failures |
| 121 | +SELECT workflow_id, workflow_name, error, p_timestamp |
| 122 | +FROM "temporal-logs" |
| 123 | +WHERE type = 'workflow' AND status = 'failed' |
| 124 | +ORDER BY p_timestamp DESC LIMIT 20; |
| 125 | + |
| 126 | +-- Activity retry hotspots |
| 127 | +SELECT activity_name, COUNT(*) as failures |
| 128 | +FROM "temporal-logs" |
| 129 | +WHERE type = 'activity' AND status = 'failed' |
| 130 | +GROUP BY activity_name ORDER BY failures DESC; |
| 131 | + |
| 132 | +-- P95 activity duration |
| 133 | +SELECT activity_name, PERCENTILE_CONT(0.95) WITHIN GROUP (ORDER BY duration_ms) as p95_ms |
| 134 | +FROM "temporal-logs" |
| 135 | +WHERE type = 'activity' AND status = 'completed' |
| 136 | +GROUP BY activity_name; |
| 137 | +``` |
| 138 | + |
| 139 | +## Development |
| 140 | + |
| 141 | +```bash |
| 142 | +pip install -e ".[dev]" |
| 143 | +pytest |
| 144 | +``` |
| 145 | + |
| 146 | +## License |
| 147 | + |
| 148 | +Apache-2.0 |
0 commit comments