Python: docs: add Vaultak runtime security integration (filter sample + guide)

[samueloladji-beep]

Summary

• Adds docs/VAULTAK_SECURITY.md — a concise integration guide explaining how to wire Vaultak into Semantic Kernel using the native filter system, with a quick-start snippet, configuration table, and event-coverage table.
• Adds python/samples/concepts/filtering/vaultak_security_filter.py — a runnable end-to-end sample (MathPlugin + TimePlugin) demonstrating both filter types.

Integration pattern: Vaultak registers as a FunctionInvocationFilter and an AutoFunctionInvocationFilter. Every plugin function call is risk-scored (0–10) before it executes; calls above the threshold raise KernelFunctionCancelledError (explicit invocations) or set context.terminate = True (auto function calling). Plugin outputs are passed through mask_pii() before they propagate to the next step.

@kernel.filter(FilterTypes.FUNCTION_INVOCATION)
async def vaultak_function_filter(context, next):
    result = vt.score_action(action=f"{plugin}-{fn}", context=dict(context.arguments))
    if result.score >= RISK_THRESHOLD:
        raise KernelFunctionCancelledError(f"[Vaultak] blocked — risk {result.score:.1f}/10")
    vt.check_policy(tool_name=..., input_data=...)
    await next(context)
    context.result._value = vt.mask_pii(str(context.result.value))

@kernel.filter(FilterTypes.AUTO_FUNCTION_INVOCATION)
async def vaultak_auto_filter(context, next):
    result = vt.score_action(action=f"{plugin}-{fn}", context={"auto_invoke": "true"})
    if result.score >= RISK_THRESHOLD:
        context.terminate = True
        return
    await next(context)

Install

pip install vaultak semantic-kernel

Get an API key at vaultak.com.

Test plan

• Run python/samples/concepts/filtering/vaultak_security_filter.py with OPENAI_API_KEY and VAULTAK_API_KEY set
• Verify MathPlugin and TimePlugin calls are intercepted by both filters
• Confirm that raising KernelFunctionCancelledError surfaces cleanly to the caller
• Review docs/VAULTAK_SECURITY.md renders correctly on GitHub

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a runnable example and documentation for integrating Vaultak runtime security with Semantic Kernel filters to risk-score, enforce policy, and mask PII during tool/function invocation.

Changes:

• Added a Python sample demonstrating FunctionInvocationFilter and AutoFunctionInvocationFilter with Vaultak scoring, policy checks, and output masking.
• Added documentation describing the integration, setup, and a “quick start” code snippet.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.

File	Description
python/samples/concepts/filtering/vaultak_security_filter.py	New end-to-end sample wiring Vaultak into SK filters and demonstrating a short conversation.
docs/VAULTAK_SECURITY.md	New integration guide explaining Vaultak + SK filters with install steps and quick start snippet.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[samueloladji-beep]

@microsoft-github-policy-service agree

[github-actions]

Automated Code Review

Reviewers: 4 | Confidence: 95%

✗ Correctness

The sample file imports KernelFunctionCancelledError from semantic_kernel.exceptions, but this class does not exist anywhere in the Semantic Kernel Python package. The only cancellation-related exception is OperationCancelledException. This will cause an ImportError at module load time, making the sample completely non-functional. The same non-existent import appears in the documentation quick-start snippet.

✓ Security Reliability

The sample and docs import KernelFunctionCancelledError which does not exist in the semantic_kernel.exceptions module (verified: only OperationCancelledException exists), causing an ImportError at load time. Additionally, the PII masking via context.result._value = ... is silently non-functional: FunctionResult is a Pydantic v2 model where the field is value, not _value; setting _value creates an unused attribute without changing the result propagated downstream. This means the advertised PII protection does not actually work.

✓ Test Coverage

This PR adds a Vaultak security filter sample and documentation but provides zero test coverage. Every other filtering sample in python/samples/concepts/filtering/ is registered in python/tests/samples/test_concepts.py as a parameterized integration test case. The new vaultak_security_filter.py is not added there, nor does it have any unit tests with mocked Vaultak dependencies. Given that vaultak is an external third-party package not in the project's dependencies, the sample cannot even be imported without it — meaning CI would fail if it were naively added to test_concepts.py. At minimum, unit tests with mocked vaultak calls should verify the filter logic (blocking on high scores, PII masking on output, terminate behavior).

✗ Design Approach

The main design issue is that the sample wires Vaultak into FUNCTION_INVOCATION, which in Semantic Kernel also wraps prompt functions. That makes this integration risk-score and potentially block kernel.invoke_prompt(...) turns before any plugin/tool call happens, which is broader than the PR describes and changes the behavior of the sample conversation itself.

Flagged Issues

• KernelFunctionCanceledError does not exist in semantic_kernel.exceptions (or anywhere in the SK Python package). The only cancellation exception is OperationCancelledException (defined in python/semantic_kernel/exceptions/kernel_exceptions.py:55). Both the sample and the docs will fail with ImportError at import time.
• vaultak_function_filter is attached at the FUNCTION_INVOCATION layer, which also covers prompt functions. Since chat() uses kernel.invoke_prompt(...) for every user turn, and invoke_prompt() creates a KernelFunctionFromPrompt that goes through self.invoke(...) (which always runs function-invocation filters), a high-risk-looking prompt can be blocked before any plugin function call or LM-selected tool call occurs. This is broader than the documented integration scope.

Suggestions

• Gate the Vaultak FUNCTION_INVOCATION filter so it only applies to real plugin/tool functions (e.g., check context.function.plugin_name is not None), or move the blocking logic for tool use to the auto-function path, so the sample matches the stated 'plugin function call / LM tool selection' scope in the documentation.

---

Automated review by samueloladji-beep's agents

[github-actions]

KernelFunctionCancelledError does not exist in semantic_kernel.exceptions. The module's __all__ exports only: KernelException, KernelFunctionAlreadyExistsError, KernelFunctionNotFoundError, KernelInvokeException, KernelPluginInvalidConfigurationError, KernelPluginNotFoundError, KernelServiceNotFoundError, OperationCancelledException. This import will raise ImportError at module load, making the sample completely non-functional.

Suggested change

	from semantic_kernel.core_plugins import MathPlugin, TimePlugin
	from semantic_kernel.exceptions import OperationCancelledException

[github-actions]

Same KernelFunctionCanceledError import issue as the sample — this class does not exist. The docs quick-start will fail at import. Use OperationCancelledException (or a custom subclass).

Suggested change

	"VAULTAK_API_KEY environment variable is not set. "
	from semantic_kernel.exceptions import OperationCancelledException

[github-actions]

Using FUNCTION_INVOCATION here makes Vaultak run on prompt functions too, not just plugin/tool calls. chat() drives every turn through kernel.invoke_prompt(...) (lines 123-129), which builds a prompt function and calls self.invoke(...) (kernel.py:248-255), always executing function-invocation filters. A risky user prompt can therefore be blocked before any plugin function call happens, which is broader than the docs describe.

[samueloladji-beep]

@dluc @craigomatic @glahaye — ready for review. This adds a Vaultak runtime security filter sample and guide using SK's FunctionInvocationFilter and AutoFunctionInvocationFilter.

[samueloladji-beep]

Hey @moonbox3, all CI checks are passing — would appreciate a review when you get a chance. Thanks!