Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
201 changes: 201 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# Testing the Auction Orchestration System

## Quick Test Summary

The auction orchestration system has been integrated into the existing Prebid endpoints. You can test it right away using the Fastly local server!

## How to Test

### 1. Start the Local Server

```bash
fastly compute serve
```

### 2. Test with Existing Endpoint

The `/third-party/ad` endpoint now uses the orchestrator when `auction.enabled = true` in config.

**Test Request:**
```bash
curl -X POST http://localhost:7676/third-party/ad \
-H "Content-Type: application/json" \
-d '{
"adUnits": [
{
"code": "header-banner",
"mediaTypes": {
"banner": {
"sizes": [[728, 90], [970, 250]]
}
}
},
{
"code": "sidebar",
"mediaTypes": {
"banner": {
"sizes": [[300, 250], [300, 600]]
}
}
}
]
}'
```

### 3. What You'll See

**With Orchestrator Enabled** (`auction.enabled = true`):
- Logs showing: `"Using auction orchestrator"`
- Parallel execution of APS (mocked) and Prebid (real)
- GAM mediation (mocked) selecting winning bids
- Final response with winning creatives

**With Orchestrator Disabled** (`auction.enabled = false`):
- Logs showing: `"Using legacy Prebid flow"`
- Direct Prebid Server call (backward compatible)

##Configuration

Edit `trusted-server.toml` to customize the auction:

```toml
# Enable/disable orchestrator
[auction]
enabled = true
strategy = "parallel_mediation" # or "parallel_only" or "waterfall"
bidders = ["prebid", "aps"]
mediator = "gam"
timeout_ms = 2000

# Mock provider configs
[integrations.aps]
enabled = true
mock = true
mock_price = 2.50

[integrations.gam]
enabled = true
mock = true
inject_house_bids = true
gam_win_rate = 30 # GAM wins 30% of the time
```

## Test Scenarios

### Scenario 1: Parallel + Mediation (Default)
**Config:**
```toml
[auction]
enabled = true
strategy = "parallel_mediation"
bidders = ["prebid", "aps"]
mediator = "gam"
```

**Expected Flow:**
1. Prebid queries real SSPs
2. APS returns mock bids ($2.50 CPM)
3. GAM mediates between all bids
4. Winning creative returned

### Scenario 2: Parallel Only (No Mediation)
**Config:**
```toml
[auction]
enabled = true
strategy = "parallel_only"
bidders = ["prebid", "aps"]
# No mediator
```

**Expected Flow:**
1. Prebid and APS run in parallel
2. Highest bid wins automatically
3. No GAM mediation

### Scenario 3: Waterfall
**Config:**
```toml
[auction]
enabled = true
strategy = "waterfall"
bidders = ["prebid", "aps"]
```

**Expected Flow:**
1. Try Prebid first
2. If Prebid returns no bids, try APS
3. Return first successful bid

### Scenario 4: Legacy Mode (Backward Compatible)
**Config:**
```toml
[auction]
enabled = false
```

**Expected Flow:**
- Original Prebid-only behavior
- No orchestration overhead

## Debugging

### Check Logs
The orchestrator logs extensively:
```
INFO: Using auction orchestrator
INFO: Running auction with strategy: parallel_mediation
INFO: Running 2 bidders in parallel
INFO: Requesting bids from: prebid
INFO: Prebid returned 2 bids (time: 120ms)
INFO: Requesting bids from: aps
INFO: APS (MOCK): returning 2 bids in 80ms
INFO: GAM mediation: slot 'header-banner' won by 'amazon-aps' at $2.50 CPM
```

### Verify Provider Registration
Look for these log messages on startup:
```
INFO: Registering auction provider: prebid
INFO: Registering auction provider: aps
INFO: Registering auction provider: gam
```

### Common Issues

**Issue:** `"Provider 'aps' not registered"`
**Fix:** Make sure `[integrations.aps]` is configured in `trusted-server.toml`

**Issue:** `"No bidders configured"`
**Fix:** Make sure `bidders = ["prebid", "aps"]` is set in `[auction]`

**Issue:** Tests fail with WASM errors
**Explanation:** Async tests don't work in WASM test environment. Integration tests via HTTP work fine!

## Next Steps

1. **Test with real Prebid Server** - Verify Prebid bids work correctly
2. **Implement real APS** - Replace mock with actual Amazon TAM API calls
3. **Implement real GAM** - Add Google Ad Manager API integration
4. **Add metrics** - Track bid rates, win rates, latency per provider

## Mock Provider Behavior

### APS (Amazon)
- Returns bids for all slots
- Default mock price: $2.50 CPM
- Always returns 2 bids
- Response time: ~80ms (simulated)

### GAM (Google Ad Manager)
- Acts as mediator
- Can inject house ads at $1.75 CPM
- Wins 30% of auctions (configurable)
- Response time: ~40ms (simulated)
- Uses hash-based "randomness" for consistent testing

### Prebid
- **Real implementation** - makes actual HTTP calls
- Queries configured SSPs
- Returns real bids from real bidders
- Response time: varies (network dependent)
3 changes: 3 additions & 0 deletions crates/common/build.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#[path = "src/error.rs"]
mod error;

#[path = "src/auction_config_types.rs"]
mod auction_config_types;

#[path = "src/settings.rs"]
mod settings;

Expand Down
Loading