Skip to content

Commit 9c9c828

Browse files
hadarishavhadarishav
authored
Merge pull request #585 from future-agi/docs/nav-cleanup
Nav cleanup: reorder quickstarts, move Release Notes
2 parents dac9712 + d074d6c commit 9c9c828

3 files changed

Lines changed: 391 additions & 6 deletions

File tree

‎src/lib/navigation.ts‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -45,13 +45,19 @@ export const tabNavigation: NavTab[] = [
4545
icon: 'rocket',
4646
href: '/docs',
4747
items: [
48-
{ title: 'Setup Observability', href: '/docs/quickstart/setup-observability' },
49-
{ title: 'Running Evals in Simulation', href: '/docs/quickstart/running-evals-in-simulation' },
50-
{ title: 'Generate Synthetic Data', href: '/docs/quickstart/generate-synthetic-data' },
5148
{ title: 'Create Prompts', href: '/docs/quickstart/prompts' },
49+
{ title: 'Generate Synthetic Data', href: '/docs/quickstart/generate-synthetic-data' },
50+
{ title: 'Running Evals in Simulation', href: '/docs/quickstart/running-evals-in-simulation' },
51+
{ title: 'Prism AI Gateway', href: '/docs/quickstart/prism-ai-gateway' },
52+
{ title: 'Setup Observability', href: '/docs/quickstart/setup-observability' },
53+
{ title: 'Annotations', href: '/docs/quickstart/annotations' },
5254
{ title: 'Setup MCP Server', href: '/docs/quickstart/setup-mcp-server' },
53-
{ title: 'Annotations Quickstart', href: '/docs/annotations/quickstart' },
54-
{ title: 'Prism AI Gateway Quickstart', href: '/docs/prism/quickstart' },
55+
]
56+
},
57+
{
58+
title: 'Release Notes',
59+
items: [
60+
{ title: 'Release Notes', href: '/docs/release-notes' },
5561
]
5662
},
5763
]
@@ -550,7 +556,6 @@ export const tabNavigation: NavTab[] = [
550556
{ title: 'Roles & Permissions', href: '/docs/roles-and-permissions' },
551557
{ title: 'Installation', href: '/docs/installation' },
552558
{ title: 'FAQ', href: '/docs/faq' },
553-
{ title: 'Release Notes', href: '/docs/release-notes' },
554559
]
555560
},
556561
{

‎src/pages/docs/quickstart/annotations.mdx‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
title: "Quickstart"
3+
description: "Get started with annotations in 5 minutes -- create a label, set up a queue, add items, and start annotating."
4+
---
5+
6+
## What you will do
7+
8+
In this walkthrough you will create an annotation label, set up a queue, add traces to it, and annotate your first item. The entire flow takes about 5 minutes.
9+
10+
<Steps>
11+
<Step title="Create an annotation label">
12+
Navigate to **Annotations** in the left sidebar, then open the **Labels** tab. Click **Create Label**.
13+
14+
![Labels page](/images/docs/annotations/labels-list.png)
15+
16+
Fill in the form:
17+
18+
| Field | Value |
19+
|-------|-------|
20+
| Name | `Sentiment` |
21+
| Type | Categorical |
22+
| Options | `Positive`, `Negative`, `Neutral` |
23+
| Allow Notes | Enabled |
24+
25+
Click **Create** to save.
26+
27+
![Create label](/images/docs/annotations/create-label-categorical.png)
28+
</Step>
29+
30+
<Step title="Create a queue">
31+
Switch to the **Queues** tab and click **Create Queue**.
32+
33+
| Field | Value |
34+
|-------|-------|
35+
| Name | `Review Queue` |
36+
| Labels | Select the `Sentiment` label you just created |
37+
| Assignment Strategy | Round Robin |
38+
| Annotators | Add yourself |
39+
| Annotations Required | 1 |
40+
41+
Click **Create** to save the queue.
42+
43+
![Create queue](/images/docs/annotations/create-queue.png)
44+
</Step>
45+
46+
<Step title="Add items to the queue">
47+
Go to your **Observe** project and open the **LLM Tracing** view. Select one or more traces using the checkboxes, then click the **Add to Queue** button in the toolbar.
48+
49+
In the dialog, choose **Review Queue** and confirm. The selected traces are now queue items with a **Pending** status.
50+
</Step>
51+
52+
<Step title="Start annotating">
53+
Go back to **Annotations > Queues** and click on **Review Queue** to open its detail page. Click **Start Annotating**.
54+
55+
The annotation workspace loads the first pending item. You will see:
56+
57+
- The trace content on the left.
58+
- The annotation panel on the right with your `Sentiment` label.
59+
60+
Select an option (e.g. **Positive**), optionally add a note, and click **Submit**.
61+
62+
![Annotation workspace](/images/docs/annotations/annotate-workspace.png)
63+
64+
The workspace automatically advances to the next item. You can also click **Skip** to move past an item you cannot annotate.
65+
</Step>
66+
67+
<Step title="Review progress">
68+
Click the **Analytics** tab on the queue detail page to see completion rates, annotator activity, and label distribution.
69+
70+
![Analytics](/images/docs/annotations/queue-detail-analytics.png)
71+
</Step>
72+
</Steps>
73+
74+
<Tip>
75+
**Keyboard shortcuts** speed up annotation significantly:
76+
77+
- **Ctrl+Enter** (or Cmd+Enter) -- Submit the current annotation
78+
- **1-9** -- Select a categorical option by its position
79+
- **S** -- Skip the current item
80+
</Tip>
81+
82+
## Next Steps
83+
84+
<CardGroup cols={2}>
85+
<Card title="Annotation Labels" icon="tags" href="/docs/annotations/features/labels">
86+
Explore all five label types and their configuration options.
87+
</Card>
88+
<Card title="Queues & Workflow" icon="list-check" href="/docs/annotations/features/queues">
89+
Configure assignment strategies, multi-annotator requirements, and review workflows.
90+
</Card>
91+
<Card title="Scores" icon="chart-simple" href="/docs/annotations/concepts/scores">
92+
Understand how annotation data is stored and queried via the Score model.
93+
</Card>
94+
</CardGroup>

0 commit comments

Comments
 (0)