Skip to content

Python: Replace Eval SDK with AI Projects SDK in evaluation sample#2540

Merged
YusakuNo1 merged 4 commits intomainfrom
users/daviwu/replace-evals-sdk-with-ai-projects-sdk
Dec 2, 2025
Merged

Python: Replace Eval SDK with AI Projects SDK in evaluation sample#2540
YusakuNo1 merged 4 commits intomainfrom
users/daviwu/replace-evals-sdk-with-ai-projects-sdk

Conversation

@YusakuNo1
Copy link
Contributor

Motivation and Context

Description

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings December 1, 2025 08:08
@markwallace-microsoft markwallace-microsoft added documentation Improvements or additions to documentation python labels Dec 1, 2025
@github-actions github-actions bot changed the title Replace Eval SDK with AI Projects SDK in evaluation sample Python: Replace Eval SDK with AI Projects SDK in evaluation sample Dec 1, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces the Azure AI Evaluation SDK with the Azure AI Projects SDK in the self-reflection evaluation sample, transitioning from the GroundednessEvaluator to the OpenAI evals API for groundedness evaluation.

Key Changes:

  • Migrated from azure-ai-evaluation SDK to azure-ai-projects SDK with OpenAI client integration
  • Replaced GroundednessEvaluator with direct OpenAI evals API calls (client.evals.create() and client.evals.runs.create())
  • Updated environment variables from AZURE_OPENAI_ENDPOINT/AZURE_OPENAI_API_KEY to AZURE_AI_PROJECT_ENDPOINT

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File Description
python/samples/getting_started/evaluation/self_reflection/self_reflection.py Replaced evaluation SDK with AI Projects SDK; refactored evaluation logic to use OpenAI evals API with polling mechanism
python/samples/getting_started/evaluation/self_reflection/README.md Updated documentation to reflect JSONL format, new SDK dependency, and new environment variable requirements
python/samples/getting_started/evaluation/self_reflection/.env.example Updated environment variable template from OpenAI endpoint/key to AI Project endpoint
Comments suppressed due to low confidence (1)

python/samples/getting_started/evaluation/self_reflection/self_reflection.py:189

  • Testing for None should use the 'is' operator.
        if eval_run_output_items == None:

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

@YusakuNo1 YusakuNo1 enabled auto-merge December 1, 2025 17:53
@YusakuNo1 YusakuNo1 added this pull request to the merge queue Dec 2, 2025
Merged via the queue into main with commit e9ab514 Dec 2, 2025
23 checks passed
@YusakuNo1 YusakuNo1 deleted the users/daviwu/replace-evals-sdk-with-ai-projects-sdk branch December 2, 2025 21:24
arisng pushed a commit to arisng/agent-framework that referenced this pull request Feb 2, 2026
…icrosoft#2540)

* Replace Eval SDK with AI Projects SDK

* Update per PR review suggestions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants