diff --git a/.env.example b/.env.example index 7b868bd..afa248b 100644 --- a/.env.example +++ b/.env.example @@ -6,6 +6,9 @@ AZURE_MODEL_ROUTER_DEPLOYMENT=model-router # Azure OpenAI endpoint (for baseline model) AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com AZURE_OPENAI_KEY=your-azure-openai-api-key +# Must match a key in the pricing section of your config file (e.g. gpt-4o, gpt-5). +# If you use a custom deployment name, add a matching entry to configs/default.yaml +# under pricing: using your exact deployment name. Otherwise baseline costs show as $0.00. AZURE_BASELINE_DEPLOYMENT=gpt-5 # Judge model (defaults to same endpoint as baseline; override to use a different model) diff --git a/QUICKSTART.md b/QUICKSTART.md index 2cd6121..76644ad 100644 --- a/QUICKSTART.md +++ b/QUICKSTART.md @@ -139,6 +139,8 @@ copy .env.example .env # Windows ``` Open `.env` and set the four endpoint URLs and API keys (router, baseline, judge, optional Foundry project). The file is in `.gitignore`, so your secrets won't be committed. +> **Note:** The value you set for `AZURE_BASELINE_DEPLOYMENT` must match a key in the `pricing` section of your config file. If you use a custom deployment name such as `gpt-4o-baseline`, add a matching entry under `pricing:` in `configs/default.yaml` with the same input/output rates. Otherwise baseline costs will show as $0.00 and the cost comparison will be inaccurate. This applies whether you are using Quick Deploy or Custom Deploy. + ### 2. Pick or edit a config - `configs/quick_test.yaml` — small, fast (~10 prompts) — good first run. - `configs/default.yaml` — full benchmark (100 prompts).