Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 53 additions & 1 deletion .agents/skills/obol-stack-dev/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ obol kubectl exec -i -n openclaw-<id> deploy/openclaw -c openclaw -- python3 - <
- Use `obol model setup --provider <name> --api-key <key>` for cloud provider config
- Wait for pod readiness AND HTTP readiness before sending inference requests
- Clean up test instances with `obol openclaw delete --force <id>` (flag BEFORE arg)
- Set env vars for dev mode: `OBOL_CONFIG_DIR`, `OBOL_BIN_DIR`, `OBOL_DATA_DIR`
- Set env vars for dev mode: `OBOL_DEVELOPMENT=true`, `OBOL_CONFIG_DIR`, `OBOL_BIN_DIR`, `OBOL_DATA_DIR`

### MUST NOT DO
- Call internal Go functions directly when testing the deployment path
Expand All @@ -189,3 +189,55 @@ obol kubectl exec -i -n openclaw-<id> deploy/openclaw -c openclaw -- python3 - <
- Assume TCP connectivity means HTTP is ready (port-forward warmup race)
- Use `app.kubernetes.io/instance=openclaw-<id>` for pod labels (Helm uses `openclaw`)
- Run multiple integration tests without cleaning up between them (pod sandbox errors)

## Sell-Side Monetize Lifecycle

### Architecture

The monetize subsystem enables pay-per-request access to local compute via x402:

```
ServiceOffer CR → monetize.py reconciliation → Middleware + HTTPRoute + pricing route
Client request ──► Traefik ──► x402-verifier (ForwardAuth) ──► backend (Ollama)
│ │
402 (no payment) 200 (valid payment)
Payment requirements Inference response
```

### Three-Layer Integration

1. **monetize.py** (OpenClaw skill) — Creates Middleware, HTTPRoute, pricing ConfigMap route
2. **x402-verifier** (ForwardAuth) — Checks X-PAYMENT header against facilitator
3. **Traefik Gateway API** — Routes traffic; requires ClusterIP backends (not ExternalName)

### Testing the Monetize Flow

```bash
# Prerequisites
obol stack up && obol agent init

# Create offer
obol sell http qwen35 --model "qwen3.5:35b" --per-request "0.001" \
--network "base-sepolia" --pay-to "0x<wallet>"

# Trigger reconciliation (or wait for heartbeat)
obol kubectl exec -n openclaw-obol-agent deploy/openclaw -c openclaw -- \
python3 /data/.openclaw/skills/monetize/scripts/monetize.py process qwen35 --namespace llm

# Verify 402
curl -X POST http://obol.stack:8080/services/qwen35/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"qwen3.5:35b","messages":[{"role":"user","content":"hi"}],"stream":false}'

# Run e2e test (with mock facilitator)
export OBOL_DEVELOPMENT=true OBOL_CONFIG_DIR=$(pwd)/.workspace/config OBOL_BIN_DIR=$(pwd)/.workspace/bin
go test -tags integration -v -run TestIntegration_PaymentGate_FullLifecycle -timeout 5m ./internal/x402/
```

### Known Gotchas

- **ExternalName services**: Traefik Gateway API rejects ExternalName as HTTPRoute backends → 500 after valid payment. Use ClusterIP+Endpoints.
- **Model pull timeout**: monetize.py checks `/api/tags` before `/api/pull` to avoid hanging on cached models.
- **Facilitator HTTPS**: URLs must be HTTPS except localhost, 127.0.0.1, host.k3d.internal, host.docker.internal.
- **ConfigMap propagation**: File watcher takes 60-120s. Force restart verifier for immediate effect.
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ go test ./...

# Integration tests (requires running cluster + Ollama + API keys)
export $(grep -v '^#' .env | xargs)
export OBOL_DEVELOPMENT=true
export OBOL_CONFIG_DIR=$(pwd)/.workspace/config
export OBOL_BIN_DIR=$(pwd)/.workspace/bin
export OBOL_DATA_DIR=$(pwd)/.workspace/data
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Tests exercise the full deployment path through `obol` CLI verbs: `obol openclaw

# Set environment
export $(grep -v '^#' .env | xargs)
export OBOL_DEVELOPMENT=true
export OBOL_CONFIG_DIR=$(pwd)/.workspace/config
export OBOL_BIN_DIR=$(pwd)/.workspace/bin
export OBOL_DATA_DIR=$(pwd)/.workspace/data
Expand Down
9 changes: 5 additions & 4 deletions .agents/skills/obol-stack-dev/references/obol-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,17 +62,18 @@ return cmd.Run()
|---------|-------------|
| `obol network list` | Show available networks |
| `obol network install <network> [flags]` | Create deployment config |
| `obol network sync <network>/<id>` | Deploy to cluster |
| `obol network delete <network>/<id> --force` | Remove deployment |
| `obol network sync [<network>[/<id>]]` | Deploy to cluster (auto-resolves: no arg, type, or type/id) |
| `obol network sync --all` | Sync all network deployments |
| `obol network delete [<network>[/<id>]]` | Remove deployment (auto-resolves: no arg, type, or type/id) |

### Application Management

| Command | Description |
|---------|-------------|
| `obol app install <chart>` | Install Helm chart as app |
| `obol app sync <app>/<id>` | Deploy to cluster |
| `obol app sync [<app>[/<id>]]` | Deploy to cluster (auto-resolves: no arg, type, or type/id) |
| `obol app list` | List installed apps |
| `obol app delete <app>/<id> --force` | Remove app |
| `obol app delete [<app>[/<id>]]` | Remove app (auto-resolves: no arg, type, or type/id) |

### Tunnel Management

Expand Down
110 changes: 110 additions & 0 deletions .github/workflows/docker-publish-x402-verifier.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: Build and Publish x402-verifier Image

on:
push:
branches:
- main
- feat/secure-enclave-inference
tags:
- 'v*'
paths:
- 'internal/x402/**'
- 'cmd/x402-verifier/**'
- 'Dockerfile.x402-verifier'
- 'go.mod'
- 'go.sum'
- '.github/workflows/docker-publish-x402-verifier.yml'
workflow_dispatch:

concurrency:
group: x402-verifier-${{ github.ref }}
cancel-in-progress: true

env:
REGISTRY: ghcr.io
IMAGE_NAME: obolnetwork/x402-verifier

jobs:
# ---------------------------------------------------------------------------
# Job 1: Build the x402-verifier binary and publish the image.
# Uses the same action versions as the working OpenClaw workflow.
# ---------------------------------------------------------------------------
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
digest: ${{ steps.build-push.outputs.digest }}

steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1

- name: Set up QEMU
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0

- name: Login to GitHub Container Registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract image metadata
id: meta
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha,prefix=
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' || github.ref == 'refs/heads/feat/secure-enclave-inference' }}
labels: |
org.opencontainers.image.title=x402-verifier
org.opencontainers.image.description=x402 payment verification sidecar for Obol Stack
org.opencontainers.image.vendor=Obol Network
org.opencontainers.image.source=https://github.com/ObolNetwork/obol-stack

- name: Build and push
id: build-push
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: Dockerfile.x402-verifier
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=x402-verifier
cache-to: type=gha,scope=x402-verifier,mode=max
provenance: true
sbom: true

# ---------------------------------------------------------------------------
# Job 2: Security scan the published image using the exact digest from build.
# ---------------------------------------------------------------------------
security-scan:
needs: build
runs-on: ubuntu-latest
permissions:
security-events: write

steps:
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@22438a435773de8c97dc0958cc0b823c45b064ac # master
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ needs.build.outputs.digest }}
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'

- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@b13d724d35ff0a814e21683638ed68ed34cf53d1 # main
with:
sarif_file: 'trivy-results.sarif'
if: always()
Loading
Loading