Python: feat: Add AgentAuditHandler for EU AI Act compliance#14042
Python: feat: Add AgentAuditHandler for EU AI Act compliance#14042kuriangeorgebinu wants to merge 1 commit into
Conversation
Addresses issue microsoft#14034 Initial implementation of AgentAuditHandler middleware for recording and tracking AI agent actions for compliance purposes. Features: - Audit trail logging for all agent actions - EU AI Act compliance support (Articles 12, 73, 9, 11) - Integration framework for AgentAudit AI service - Compliance report generation capability - Immutable audit record structure This is a foundational implementation open for community feedback on: - Design approach for the audit handler interface - Integration points with agent execution pipeline - Compliance requirements mapping to implementation - Blockchain audit trail integration strategy Co-authored-by: GitHub Copilot
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Introduces a new AgentAuditHandler scaffold under semantic_kernel.agents.audit intended to provide an EU AI Act compliance audit trail for agent actions. The core logging and reporting logic is not yet implemented (TODOs).
Changes:
- Adds a new
auditsubpackage undersemantic_kernel.agents. - Introduces
AgentAuditHandlerclass withinitialize,log_agent_action,generate_compliance_report, andclosemethods (mostly stubs). - Exports
AgentAuditHandlerfrom the package's__init__.py.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| python/semantic_kernel/agents/audit/init.py | Creates the new audit package and exposes AgentAuditHandler. |
| python/semantic_kernel/agents/audit/agent_audit_handler.py | Adds the new handler class with placeholder implementations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| async def log_agent_action( | ||
| self, | ||
| agent_name: str, | ||
| action_type: str, | ||
| action_details: dict[str, Any], | ||
| timestamp: Optional[str] = None, | ||
| ) -> None: |
| "action_details": action_details, | ||
| "timestamp": timestamp, | ||
| } | ||
|
|
||
| logger.debug(f"Recording audit entry: {audit_record}") |
| logger.info(f"Initializing AgentAuditHandler with service: {self.service_url}") | ||
| self._validated = True |
| import logging | ||
| from typing import Any, Optional |
| agent_name: str, | ||
| action_type: str, | ||
| action_details: dict[str, Any], | ||
| timestamp: Optional[str] = None, |
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class AgentAuditHandler: |
|
@kuriangeorgebinu please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 91%
✓ Correctness
The existing unresolved review comments comprehensively cover the significant correctness issues in this PR (timestamp not defaulted despite docstring claim, initialize() not validating connectivity, log_agent_action being a misleading no-op, sensitive data logging, Optional style inconsistency, and missing tests). Beyond those already-flaged issues, I found no additional correctness bugs. The code is internally consistent for a stub implementation, but the already-identified issues remain valid blockers for a compliance-oriented component.
✓ Security Reliability
The primary new security concern (beyond the already-flagged issues) is that the api_key is stored as a plain string attribute, contradicting the established codebase pattern of using Pydantic's SecretStr. This means the credential can be exposed via object repr, dict access, serialization, or any inadvertent logging of the handler instance — particularly problematic for a module whose purpose is security compliance.
✓ Test Coverage
This PR adds a new AgentAuditHandler class with zero accompanying unit tests. The repository has a well-established pattern of unit tests under python/tests/unit/agents/ for every agent module (e.g., test_agent.py, test_bedrock_agent.py, test_azure_ai_agent.py). The new audit module ships without any test coverage, meaning all code paths (initialize success/failure, log_agent_action when disabled/uninitialized/valid, generate_compliance_report when disabled/uninitialized/valid, and close) are untested. This is a blocking gap for a compliance-oriented component where correctness is critical.
✗ Design Approach
The main design issue is that this PR introduces a standalone audit helper, but it does not integrate with the agent runtime’s existing interception pipeline, so it cannot deliver the stated goal of auditing all agent actions on the normal execution path. I did not find other design findings that met the repo’s evidence bar without duplicating existing review comments.
Flagged Issues
- The new audit handler is not integrated with the runtime-wide
InterventionHandlerhook that already intercepts every send/publish/response inInProcessRuntime. The repo's established pattern (python/semantic_kernel/agents/runtime/core/intervention.py:29-39) is explicitly designed for components that "modify, log or drop messages." BecauseAgentAuditHandleris a standalone class, normal agent traffic will never reach it automatically, and the stated goal of auditing all agent actions is not achieved by the current design.
Automated review by kuriangeorgebinu's agents
| """ | ||
| self.service_url = service_url | ||
| self.api_key = api_key | ||
| self.enabled = enabled |
There was a problem hiding this comment.
Security: api_key is stored as a plain str attribute, which means it can leak via repr(), __dict__, serialization, or accidental logging of this object. The rest of the SK Python codebase (e.g., openai_assistant_agent.py) uses Pydantic's SecretStr to wrap API keys. For a compliance-oriented module this is especially important—store as SecretStr and call .get_secret_value() only when transmitting to the service.
| self.enabled = enabled | |
| self.api_key: SecretStr | None = SecretStr(api_key) if api_key else None |
| raise ValueError("api_key is required when audit logging is enabled") | ||
|
|
||
| logger.info(f"Initializing AgentAuditHandler with service: {self.service_url}") | ||
| self._validated = True |
There was a problem hiding this comment.
The service_url is logged at INFO level. If a user supplies a URL with embedded credentials or tokens as query parameters (e.g., https://service.example.com?token=secret), those will be written to application logs. Consider logging only the scheme+host or omitting the URL entirely.
| self._validated = True | |
| logger.info("AgentAuditHandler initialized successfully") |
Addresses issue #14034
Initial implementation of AgentAuditHandler middleware for recording and tracking AI agent actions for compliance purposes. Features:
Motivation and Context
Description
Contribution Checklist