[AutoPR azure-ai-projects]-generated-from-SDK Generation - Python-6070083#45950
Draft
[AutoPR azure-ai-projects]-generated-from-SDK Generation - Python-6070083#45950
Conversation
This reverts commit 86d53c5.
…allow_preview=True" is not specified (#45600)
…#45611) * marking finetuning pause and resume operations as live extended tests * updating recording --------- Co-authored-by: Jayesh Tanna <jatanna@microsoft.com>
* rename env vars * rename env var * resolved comments * remove chat completion * resolved comment
* Add CSV and synthetic data generation evaluation samples Add two new evaluation samples under sdk/ai/azure-ai-projects/samples/evaluations/: - sample_evaluations_builtin_with_csv.py: Demonstrates evaluating pre-computed responses from a CSV file using the csv data source type. Uploads a CSV file via the datasets API, runs coherence/violence/f1 evaluators, and polls results. - sample_synthetic_data_evaluation.py: Demonstrates synthetic data evaluation (preview) that generates test queries from a prompt, sends them to a model target, and evaluates responses with coherence/violence evaluators. Also adds: - data_folder/sample_data_evaluation.csv: Sample CSV data file with 3 rows - README.md: Updated sample index with both new samples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update synthetic eval sample: agent target + cleaner dataset ID retrieval - Switch from model target to agent target (azure_ai_agent) - Create agent version via agents.create_version() before evaluation - Simplify output_dataset_id retrieval using getattr instead of nested hasattr/isinstance checks - Add AZURE_AI_AGENT_NAME env var requirement - Remove input_messages (not needed for agent target) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add model target synthetic eval sample, cross-reference both - Add sample_synthetic_data_model_evaluation.py for model target with input_messages system prompt - Update sample_synthetic_data_evaluation.py docstring with cross-reference - Update README.md with both synthetic samples (agent and model) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename synthetic agent sample, clarify README, add prompt/files comments - Rename sample_synthetic_data_evaluation.py to sample_synthetic_data_agent_evaluation.py - Clarify README: JSONL dataset vs CSV dataset descriptions - Remove (preview) from synthetic sample descriptions in README - Add comments about prompt and reference_files options in both synthetic samples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Skip new eval samples in recording tests Add sample_evaluations_builtin_with_csv.py, sample_synthetic_data_agent_evaluation.py, and sample_synthetic_data_model_evaluation.py to samples_to_skip list since they require file upload prerequisites or are long-running preview features. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename env vars per PR review: FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL_NAME Address review comments from howieleung: - AZURE_AI_PROJECT_ENDPOINT -> FOUNDRY_PROJECT_ENDPOINT - AZURE_AI_MODEL_DEPLOYMENT_NAME -> FOUNDRY_MODEL_NAME Updated in all 3 new samples and README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename AZURE_AI_AGENT_NAME to FOUNDRY_AGENT_NAME per review Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update changelog with new sample entries Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ollingMethod` should be private (#45685)
…ed when relevant (#45718)
* New samples * resolved comments
* LLM validation use 5.2 and chat completion * change log * Resolved comments
…Azure/azure-sdk-for-python into feature/azure-ai-projects/2.0.2
* Adding-Upload-Evaluator * Adding-Upload-Evaluator * Adding-Upload-Evaluator * Adding-Upload-Evaluator-aio * rename * added - eval and eval run * fix * adding tests * updated as per review
* added toolset samples and more * improved samples * resolved comment
…lity gates (#45854) Additional Pyright and Pylint errors will need to be fixed in another PR.
* Fix pyright and pylint errors in evaluator upload operations - Replace hasattr(e, 'error_code') with getattr(e, 'error_code', None) to fix pyright reportAttributeAccessIssue on HttpResponseError - Narrow union types with isinstance checks for EvaluatorVersion and CodeBasedEvaluatorDefinition to fix pyright type errors on definition/blob_uri - Extract _upload_folder_to_blob() and _set_blob_uri() helpers to fix pylint R0914 (too many local variables) in upload() - Update tests to use MagicMock(spec=CodeBasedEvaluatorDefinition) for definition mocks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add missing docstring params, return, and rtype for Azure SDK lint rules Fix C4739/C4741/C4742 violations from azure-pylint-guidelines-checker: - _set_blob_uri: add :param and :type for evaluator_version, blob_uri - _start_pending_upload_and_get_container_client: add :param/:type/:return/:rtype - _get_next_version: add :param/:type/:return/:rtype Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add missing param docs for _upload_folder_to_blob (C4739) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…m public methods (#45834)
…uild of package @azure-tools/typespec-client-generator-core) (#45887)
…ys names as they are now package-internal (#45938)
…g.yaml', API Version: v1, SDK Release Type: stable, and CommitSHA: '57e7a9ebf3c779e4f8c90b9e140bf09c14aa0830' in SpecRepo: 'https://github.com/Azure/azure-rest-api-specs' Pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6069658 Refer to https://eng.ms/docs/products/azure-developer-experience/develop/sdk-release/sdk-release-prerequisites to prepare for SDK release.
…tspconfig.yaml', API Version: v1, SDK Release Type: stable, and CommitSHA: '57e7a9ebf3c779e4f8c90b9e140bf09c14aa0830' in SpecRepo: 'https://github.com/Azure/azure-rest-api-specs' Pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6069658 Refer to https://eng.ms/docs/products/azure-developer-experience/develop/sdk-release/sdk-release-prerequisites to prepare for SDK release." This reverts commit 39fb228.
…g.yaml', API Version: v1, SDK Release Type: stable, and CommitSHA: 'bd70f1fbb6f1691a3107d791c7aa5d1dede46f01' in SpecRepo: 'https://github.com/Azure/azure-rest-api-specs' Pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6070083 Refer to https://eng.ms/docs/products/azure-developer-experience/develop/sdk-release/sdk-release-prerequisites to prepare for SDK release.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Configurations: 'specification/ai-foundry/data-plane/Foundry/tspconfig.yaml', API Version: v1, SDK Release Type: stable, and CommitSHA: 'bd70f1fbb6f1691a3107d791c7aa5d1dede46f01' in SpecRepo: 'https://github.com/Azure/azure-rest-api-specs' Pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6070083 Refer to https://eng.ms/docs/products/azure-developer-experience/develop/sdk-release/sdk-release-prerequisites to prepare for SDK release.