Phase 5: Dynamic Webhook Middleware Kubernetes Controller

[Sanskarzz]

[WIP] Depends on the merge of Phase 3 PR and 4 PR

Summary

This PR implements the fifth phase of the dynamic webhook middleware configuration system (RFC THV-0017), introducing Kubernetes custom resource definitions (CRDs), their respective controller reconciling mechanisms, and integration into the core MCPServer lifecycle.

Fixes #3401

Key Changes

1. MCPWebhookConfig CRD Creation

   • Introduced MCPWebhookConfig CRD in api/v1alpha1 matching the specifications described in RFC THV-0017.
   • Allows users to declaratively specify sets of Validating and Mutating webhooks.
   • Includes full configuration for security integrations:
     • HMACSecretRef for signing request payloads.
     • TLSConfig (CA, Client Cert, and Key secrets) for rigorous mTLS connections.
   • Fix: Updated CRD markers to use lowercase fail/ignore for FailurePolicy to align with the runner's runtime validation requirements.

2. Controller Logic and Finalizers

   • Created the MCPWebhookConfigReconciler in cmd/thv-operator/controllers/.
   • The controller manages .Status.ConfigHash calculating changes to the configuration.
   • Cross-references incoming configurations dynamically by injecting finalizers. It correctly tracks all referencing MCPServers via .Status.ReferencingServers.
   • Integrated safety guards preventing the deletion of an MCPWebhookConfig while actively referenced by an MCPServer.

3. MCPServer Controller Integration

   • Embedded WebhookConfigRef natively into MCPServerSpec.
   • Updated MCPServerStatus to explicitly trace configuration hashes linked via annotation hooks.
   • Adapted the Pod Environment builder (deploymentNeedsUpdate) to trace webhook Secret updates.
   • Upgraded createRunConfigFromMCPServer to evaluate and translate webhook settings locally using newly extracted utility functions in pkg/controllerutil/webhook.go.
   • Fix: Implemented robust lowercasing of FailurePolicy in buildWebhookConfig to ensure compatibility with the thv-proxyrunner, regardless of the case used in the CRD.

4. Testing and Verification

   • Added robust unit test coverage confirming behavior for mcpwebhookconfig_types_test.go, the controller logic (mcpwebhookconfig_controller_test.go), and utilities (webhook_test.go).
   • Introduced comprehensive end-to-end chainsaw tests ensuring valid configurations proceed through creation securely, rejecting any malformed specs early on with CEL validation endpoints.

Type of change

• Bug fix
• New feature
• Refactoring (no behavior change)
• Dependency update
• Documentation
• Other (describe):

Test plan

• Unit tests (task test)
• E2E tests (task test-e2e)
• Linting (task lint-fix)
• Manual testing (describe below)

Manual Verification

Manual testing was performed using a local Kind cluster and the fetch MCPServer.

1. Setup:
   • Deployed the operator using task operator-deploy-local.
   • Deployed an echo webhook server: kubectl apply -f manual-testing-phase5/echo-server.yaml.

    spec:
      containers:
      - name: echo
        image: ealen/echo-server:latest
   

2. Configuration:
   • Created an MCPWebhookConfig pointing to the echo server with insecureSkipVerify: true.
   • Created a fetch MCPServer referencing the config.
3. Execution:
   • Verified that the operator successfully reconciled the MCPWebhookConfig and generated a configHash.
   • Verified that the fetch server picked up the configuration and started the thv-proxyrunner.
   • Result: Inspected the fetch pod logs and confirmed that the mutating webhook middleware was active and correctly invoking the echo server (resulting in "denied request" logs as expected since the echo server doesn't return a valid allowed: true response).
4. Dynamic Updates:
   • Updated the MCPWebhookConfig (e.g., changed the failure policy or URL).
   • Verified that the operator detected the change and restarted the fetch pod automatically to load the new settings.