feat(eval): simplify template variables — drop _text suffix, align with industry patterns#839
Merged
feat(eval): simplify template variables — drop _text suffix, align with industry patterns#839
Conversation
Deploying agentv with
|
| Latest commit: |
f29d165
|
| Status: | ⚡️ Build in progress... |
This was referenced Mar 29, 2026
…th industry patterns
- {{output}}, {{input}}, {{expected_output}} now resolve to human-readable
text instead of JSON.stringify'd message arrays
- Deprecated _text aliases ({{input_text}}, {{output_text}},
{{expected_output_text}}) still work but emit a stderr warning
- Removed outputText, inputText, expectedOutputText from CodeGraderInput
schema — code graders should extract text from Message.content using
getTextContent() from @agentv/core
- Removed EnrichedCodeGraderInput type (no longer needed)
- Updated default evaluator template to use new variable names
- Updated prompt-validator to accept both new and deprecated variable names
Closes #825
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace deprecated _text suffix template variables with their canonical equivalents: output_text→output, input_text→input, expected_output_text→expected_output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The hasExpectedOutput check was missing the deprecated alias
EXPECTED_OUTPUT_TEXT, causing templates using only {{ expected_output_text }}
to fail validation with 'Missing required fields'.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…emoved text fields The outputText, inputText, and expectedOutputText fields were removed from CodeGraderInput. Update all example code graders to extract text from Message arrays directly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…removed outputText The outputText field was removed from CodeGraderInput. Update the agentv create assertion template to extract text from output Message arrays using a getMessageText() helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
christso
force-pushed
the
feat/825-template-vars
branch
from
e822d86 to
f29d165
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #825
Summary
Simplifies LLM grader template variables and code grader wire format:
{{output}},{{input}},{{expected_output}}now resolve to human-readable text (was JSON.stringify'd message arrays)_textaliases ({{input_text}},{{output_text}},{{expected_output_text}}) still work but emit a stderr warningoutputText,inputText,expectedOutputTextfromCodeGraderInputschema — code graders should extract text fromMessage.contentusinggetTextContent()EnrichedCodeGraderInputtype (no longer needed)Breaking Changes
CodeGraderInputno longer hasoutputText,inputText,expectedOutputTextfieldsEnrichedCodeGraderInputtype removedoutput/inputMessage arrays directlyDepends on
feat/817-content-union)