-
Notifications
You must be signed in to change notification settings - Fork 12
Create prompt templates for agent communication (CS-10477) #4230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
habdelra
merged 7 commits into
main
from
cs-10477-create-prompt-templates-for-agent-communication
Mar 24, 2026
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
a27eb17
Define FactoryAgent interface and OpenRouter integration (CS-10476)
habdelra 7599ddb
Address PR review feedback: lint fixes and code improvements
habdelra 7a3980d
Create prompt templates for agent communication (CS-10477)
habdelra 56443fc
Address PR review feedback: tool results, outputFormat, and template …
habdelra 0a9d061
Thread previousActions and iteration through plan() via AgentContext
habdelra e802589
Merge remote-tracking branch 'origin/main' into cs-10477-create-promp…
habdelra 48c2475
Merge remote-tracking branch 'origin/main' into cs-10477-create-promp…
habdelra File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| experiment_1/ | ||
| playwright-report/ | ||
| prompts/ | ||
| test-results/ | ||
| .eslintcache |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| ```json | ||
| [ | ||
| { | ||
| "type": "create_file | update_file | create_test | update_test | update_ticket | create_knowledge | invoke_tool | request_clarification | done", | ||
| "path": "(string, required for file actions) relative path within the realm", | ||
| "content": "(string, required for file actions) full file content", | ||
| "realm": "(\"target\" | \"test\") which realm this action targets", | ||
| "tool": "(string, required for invoke_tool) name of the tool to invoke", | ||
| "toolArgs": "(object, optional for invoke_tool) arguments to pass to the tool" | ||
| } | ||
| ] | ||
| ``` | ||
|
|
||
| ## Action Types | ||
|
|
||
| - **create_file** — Create a new card definition (.gts) or card instance (.json) in a realm. Requires `path`, `content`, and `realm`. | ||
| - **update_file** — Replace the content of an existing file. Requires `path`, `content`, and `realm`. | ||
| - **create_test** — Create a new test spec in the test realm. Requires `path`, `content`, and `realm` (must be `"test"`). | ||
| - **update_test** — Update an existing test spec. Requires `path`, `content`, and `realm` (must be `"test"`). | ||
| - **update_ticket** — Update the current ticket with notes or status changes. Optional `content` for notes. | ||
| - **create_knowledge** — Create a knowledge article for the project. Requires `path` and `content`. | ||
| - **invoke_tool** — Run a registered tool. Requires `tool` (tool name). Optional `toolArgs` (object of key-value arguments). | ||
| - **request_clarification** — Signal that you cannot proceed. Requires `content` explaining what is blocked. | ||
| - **done** — Signal that all work for this ticket is complete and tests are passing. No other fields needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # Example: Creating a Card Definition + Instance | ||
|
|
||
| ## Input | ||
|
|
||
| Ticket: "Create a StickyNote card with title and body fields" | ||
|
|
||
| ## Expected Output | ||
|
|
||
| ```json | ||
| [ | ||
| { | ||
| "type": "invoke_tool", | ||
| "tool": "search-realm", | ||
| "toolArgs": { "query": "StickyNote", "realm": "target" } | ||
| }, | ||
| { | ||
| "type": "create_file", | ||
| "path": "sticky-note.gts", | ||
| "realm": "target", | ||
| "content": "import { contains, field, CardDef, Component } from 'https://cardstack.com/base/card-api';\nimport StringField from 'https://cardstack.com/base/string';\nimport MarkdownField from 'https://cardstack.com/base/markdown';\n\nexport class StickyNote extends CardDef {\n static displayName = 'Sticky Note';\n @field title = contains(StringField);\n @field body = contains(MarkdownField);\n\n static fitted = class Fitted extends Component<typeof StickyNote> {\n <template>\n <div class='sticky-note'>\n <h3>{{@model.title}}</h3>\n <p>{{@model.body}}</p>\n </div>\n <style scoped>\n .sticky-note { padding: 1rem; background: #fef3c7; border-radius: 8px; }\n h3 { margin: 0 0 0.5rem; }\n </style>\n </template>\n };\n}\n" | ||
| }, | ||
| { | ||
| "type": "create_file", | ||
| "path": "StickyNote/welcome-note.json", | ||
| "realm": "target", | ||
| "content": "{\n \"data\": {\n \"type\": \"card\",\n \"attributes\": {\n \"title\": \"Welcome\",\n \"body\": \"This is a sample sticky note.\"\n },\n \"meta\": {\n \"adoptsFrom\": {\n \"module\": \"../sticky-note\",\n \"name\": \"StickyNote\"\n }\n }\n }\n}\n" | ||
| }, | ||
| { | ||
| "type": "create_test", | ||
| "path": "TestSpec/sticky-note.spec.ts", | ||
| "realm": "test", | ||
| "content": "import { test, expect } from '@playwright/test';\n\ntest('StickyNote renders title and body', async ({ page }) => {\n // test implementation\n});\n" | ||
| } | ||
| ] | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Example: Generating a Test Spec | ||
|
|
||
| ## Input | ||
|
|
||
| Ticket: "Create tests for the StickyNote card" | ||
|
|
||
| Implementation files already exist: | ||
|
|
||
| - `sticky-note.gts` (target realm) — defines StickyNote with title and body fields | ||
| - `StickyNote/welcome-note.json` (target realm) — sample instance | ||
|
|
||
| ## Expected Output | ||
|
|
||
| ```json | ||
| [ | ||
| { | ||
| "type": "create_test", | ||
| "path": "TestSpec/sticky-note.spec.ts", | ||
| "realm": "test", | ||
| "content": "import { test, expect } from '@playwright/test';\nimport { setupFactoryTest } from '../fixtures';\n\ntest.describe('StickyNote', () => {\n test('renders fitted view with title and body', async ({ page }) => {\n await page.goto('/StickyNote/welcome-note');\n await expect(page.locator('.sticky-note h3')).toHaveText('Welcome');\n await expect(page.locator('.sticky-note p')).toContainText('sample sticky note');\n });\n\n test('card instance has correct adoptsFrom', async ({ page }) => {\n const response = await page.request.get('/StickyNote/welcome-note.json');\n const json = await response.json();\n expect(json.data.meta.adoptsFrom.name).toBe('StickyNote');\n });\n});\n" | ||
| } | ||
| ] | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # Example: Fixing Code After Test Failure | ||
|
|
||
| ## Input | ||
|
|
||
| Previous actions created `sticky-note.gts` and a test spec. The test failed: | ||
|
|
||
| ``` | ||
| Failure: renders fitted view with title and body | ||
| Error: Timed out 5000ms waiting for expect(locator).toHaveText("Welcome") | ||
| Locator: page.locator('.sticky-note h3') | ||
| ``` | ||
|
|
||
| ## Expected Output | ||
|
|
||
| The agent identifies that the selector is wrong (the component uses a different class name) and fixes the implementation: | ||
|
|
||
| ```json | ||
| [ | ||
| { | ||
| "type": "invoke_tool", | ||
| "tool": "search-realm", | ||
| "toolArgs": { "query": "StickyNote", "realm": "target" } | ||
| }, | ||
| { | ||
| "type": "update_file", | ||
| "path": "sticky-note.gts", | ||
| "realm": "target", | ||
| "content": "import { contains, field, CardDef, Component } from 'https://cardstack.com/base/card-api';\nimport StringField from 'https://cardstack.com/base/string';\nimport MarkdownField from 'https://cardstack.com/base/markdown';\n\nexport class StickyNote extends CardDef {\n static displayName = 'Sticky Note';\n @field title = contains(StringField);\n @field body = contains(MarkdownField);\n\n static fitted = class Fitted extends Component<typeof StickyNote> {\n <template>\n <div class='sticky-note'>\n <h3 data-test-title>{{@model.title}}</h3>\n <p data-test-body>{{@model.body}}</p>\n </div>\n <style scoped>\n .sticky-note { padding: 1rem; background: #fef3c7; border-radius: 8px; }\n h3 { margin: 0 0 0.5rem; }\n </style>\n </template>\n };\n}\n" | ||
| }, | ||
| { | ||
| "type": "update_test", | ||
| "path": "TestSpec/sticky-note.spec.ts", | ||
| "realm": "test", | ||
| "content": "import { test, expect } from '@playwright/test';\n\ntest.describe('StickyNote', () => {\n test('renders fitted view with title and body', async ({ page }) => {\n await page.goto('/StickyNote/welcome-note');\n await expect(page.locator('[data-test-title]')).toHaveText('Welcome');\n await expect(page.locator('[data-test-body]')).toContainText('sample sticky note');\n });\n});\n" | ||
| } | ||
| ] | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Role | ||
|
|
||
| You are a software factory agent. You implement Boxel cards and tests in | ||
| target realms based on ticket descriptions and project context. | ||
|
|
||
| # Output Format | ||
|
|
||
| You must respond with a JSON array of actions. Each action matches this schema: | ||
|
|
||
| {{actionSchema}} | ||
|
|
||
| Respond with ONLY the JSON array. No prose, no explanation, no markdown fences | ||
| around the JSON. The orchestrator parses your response as JSON directly. | ||
|
|
||
| # Rules | ||
|
|
||
| - Every ticket must include at least one `create_test` or `update_test` action. | ||
| - Test specs go in the test realm. Implementation goes in the target realm. | ||
| - Use `invoke_tool` to search for existing cards, check realm state, or run | ||
| commands before creating files. Do not guess at existing state. | ||
| - If you cannot proceed, return a single `request_clarification` action | ||
| explaining what is blocked. | ||
| - When all work for the ticket is complete and tests are passing, return a | ||
| single `done` action. | ||
|
|
||
| # Realms | ||
|
|
||
| - Target realm: {{targetRealmUrl}} | ||
| - Test realm: {{testRealmUrl}} | ||
|
|
||
| {{#each skills}} | ||
|
|
||
| # Skill: {{name}} | ||
|
|
||
| {{content}} | ||
|
|
||
| {{#each references}} | ||
|
|
||
| ### Reference: {{.}} | ||
|
|
||
| {{/each}} | ||
| {{/each}} | ||
|
|
||
| {{#each tools}} | ||
|
|
||
| # Tool: {{name}} | ||
|
|
||
| {{description}} | ||
|
|
||
| Category: {{category}} | ||
| Output format: {{outputFormat}} | ||
|
|
||
| {{#each args}} | ||
| - {{name}} ({{type}}, {{#if required}}required{{else}}optional{{/if}}): {{description}} | ||
| {{/each}} | ||
| {{/each}} | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # Project | ||
|
|
||
| {{project.objective}} | ||
|
|
||
| {{#if project.successCriteria}} | ||
| Success criteria: | ||
| {{#each project.successCriteria}} | ||
| - {{.}} | ||
| {{/each}} | ||
| {{/if}} | ||
|
|
||
| # Knowledge | ||
|
|
||
| {{#each knowledge}} | ||
|
|
||
| ## {{title}} | ||
|
|
||
| {{content}} | ||
| {{/each}} | ||
|
|
||
| # Current Ticket | ||
|
|
||
| ID: {{ticket.id}} | ||
| Summary: {{ticket.summary}} | ||
| Status: {{ticket.status}} | ||
| Priority: {{ticket.priority}} | ||
|
|
||
| Description: | ||
| {{ticket.description}} | ||
|
|
||
| {{#if ticket.checklist}} | ||
| Checklist: | ||
| {{#each ticket.checklist}} | ||
| - [ ] {{.}} | ||
| {{/each}} | ||
| {{/if}} | ||
|
|
||
| {{#if toolResults}} | ||
|
|
||
| # Tool Results | ||
|
|
||
| You previously invoked the following tools. Use these results to inform your implementation. | ||
|
|
||
| {{#each toolResults}} | ||
|
|
||
| ## {{tool}} (exit code: {{exitCode}}) | ||
|
|
||
| ```{{outputFormat}} | ||
| {{output}} | ||
| ``` | ||
|
|
||
| {{/each}} | ||
| {{/if}} | ||
|
|
||
| # Instructions | ||
|
|
||
| Implement this ticket. Return actions that: | ||
|
|
||
| 1. Create or update card definitions (.gts) and/or card instances (.json) in the target realm | ||
| 2. Create test specs (.spec.ts) in the test realm that verify your implementation | ||
| 3. Use `invoke_tool` actions to inspect existing realm state before creating files | ||
|
|
||
| Start with the smallest working implementation, then add the test. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Project | ||
|
|
||
| {{project.objective}} | ||
|
|
||
| # Current Ticket | ||
|
|
||
| ID: {{ticket.id}} | ||
| Summary: {{ticket.summary}} | ||
|
|
||
| Description: | ||
| {{ticket.description}} | ||
|
|
||
| # Previous Attempt (iteration {{iteration}}) | ||
|
|
||
| You previously produced the following actions for this ticket: | ||
|
|
||
| {{#each previousActions}} | ||
|
|
||
| ## {{type}}: {{path}} ({{realm}} realm) | ||
|
|
||
| ``` | ||
| {{content}} | ||
| ``` | ||
|
|
||
| {{/each}} | ||
|
|
||
| # Test Results | ||
|
|
||
| The orchestrator applied your actions and ran tests. They failed. | ||
|
|
||
| Status: {{testResults.status}} | ||
| Passed: {{testResults.passedCount}} | ||
| Failed: {{testResults.failedCount}} | ||
| Duration: {{testResults.durationMs}}ms | ||
|
|
||
| {{#each testResults.failures}} | ||
|
|
||
| ## Failure: {{testName}} | ||
|
|
||
| ``` | ||
| {{error}} | ||
| ``` | ||
|
|
||
| {{#if stackTrace}} | ||
| Stack trace: | ||
|
|
||
| ``` | ||
| {{stackTrace}} | ||
| ``` | ||
|
|
||
| {{/if}} | ||
| {{/each}} | ||
|
|
||
| {{#if toolResults}} | ||
|
|
||
| # Tool Results From Previous Iteration | ||
|
|
||
| {{#each toolResults}} | ||
|
|
||
| ## {{tool}} (exit code: {{exitCode}}) | ||
|
|
||
| ```{{outputFormat}} | ||
| {{output}} | ||
| ``` | ||
|
|
||
| {{/each}} | ||
| {{/if}} | ||
|
|
||
| # Instructions | ||
|
|
||
| Fix the failing tests. You may: | ||
|
|
||
| - Update implementation files (use `update_file` actions) | ||
| - Update test specs (use `update_test` actions) | ||
| - Invoke tools to inspect current realm state | ||
| - If the test expectation is wrong, fix the test | ||
| - If the implementation is wrong, fix the implementation | ||
|
|
||
| Return the actions needed to make all tests pass. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Test Generation | ||
|
|
||
| You implemented the following files for ticket {{ticket.id}}: | ||
|
|
||
| {{#each implementedFiles}} | ||
|
|
||
| ## {{path}} ({{realm}} realm) | ||
|
|
||
| ``` | ||
| {{content}} | ||
| ``` | ||
|
|
||
| {{/each}} | ||
|
|
||
| Now generate Playwright test specs that verify this implementation. | ||
|
|
||
| Tests must: | ||
|
|
||
| - Live in the test realm as test spec files | ||
| - Import from the test fixtures and use the factory test harness | ||
| - Verify that card instances render correctly (fitted, isolated, embedded views) | ||
| - Verify card-specific behavior, field values, and relationships | ||
| - Be runnable by the `run-realm-tests` tool | ||
|
|
||
| Return only `create_test` actions. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where does this come from? I ran with
BOXEL_ENVIRONMENT=helloso I’d expect to seehttp://realm-server.hello.localhost/user/personalbut I gothttp://localhost:4201/user/personal/but I know I’m probably the only one using this at the moment, so when the time comes, I’ll add support for this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the target realm is the realm that you specified to create the project in. i don't think the software factory is aware of the boxel environment work. I can add a ticket for that though