Skip to content

CM-64735 - Reduce ai-guardrails scan latency#472

Merged
Ilanlido merged 1 commit into
mainfrom
CM-64735-ai-guardrails-scan-perf
Jun 8, 2026
Merged

CM-64735 - Reduce ai-guardrails scan latency#472
Ilanlido merged 1 commit into
mainfrom
CM-64735-ai-guardrails-scan-perf

Conversation

@Ilanlido

@Ilanlido Ilanlido commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Cluster of CLI changes that take a 0-detection ai-guardrails scan from ~5.5s to ~1.55s measured via the packaged onedir with warm tokens:

  • Reuse a process-wide requests.Session so TCP+TLS handshakes amortize across the multiple API calls per scan (was one-shot per request on macOS/Linux, paying ~300-450ms of handshake on each call).
  • Argv-peek lazy subapp registration: import only the invoked subapp at startup instead of the full set, skipping ~300ms of unrelated imports on hot paths.
  • Skip get_detection_rules when the scan returned zero detections (the common case for ai-guardrails hooks).
  • Skip POST /cli-scan/{id}/status on sync flows where the /sync response already returned the full result inline.
  • Share access tokens across CycodeClientBase instances by re-reading the on-disk cache before doing the HTTP refresh, so the ai-security client doesn't duplicate work the scan client just did.
  • Skip version-checker on the ai-guardrails scan hot path (it emits JSON to stdout — an upgrade notice would corrupt the response, plus the PyPI round-trip is a ~500ms cache-miss cost).
  • Send cli_start_time in /sync so the server can compute honest end-to-end execution_time independent of CLI-side wall clock.
  • Add coverage for the argv-peek invariant: root options that take a value must be registered so argv-peek skips past their values.

@Ilanlido Ilanlido enabled auto-merge (squash) June 8, 2026 11:58
@Ilanlido Ilanlido disabled auto-merge June 8, 2026 12:13
Cluster of CLI changes that take a 0-detection ai-guardrails scan from
~5.5s to ~1.55s measured via the packaged onedir with warm tokens:

- Reuse a process-wide requests.Session so TCP+TLS handshakes amortize
  across the multiple API calls per scan (was one-shot per request on
  macOS/Linux, paying ~300-450ms of handshake on each call).
- Argv-peek lazy subapp registration: import only the invoked subapp
  at startup instead of the full set, skipping ~300ms of unrelated
  imports on hot paths.
- Skip get_detection_rules when the scan returned zero detections (the
  common case for ai-guardrails hooks).
- Skip POST /cli-scan/{id}/status on sync flows where the /sync response
  already returned the full result inline.
- Share access tokens across CycodeClientBase instances by re-reading
  the on-disk cache before doing the HTTP refresh, so the ai-security
  client doesn't duplicate work the scan client just did.
- Skip version-checker on the ai-guardrails scan hot path (it emits
  JSON to stdout — an upgrade notice would corrupt the response, plus
  the PyPI round-trip is a ~500ms cache-miss cost).
- Send cli_start_time in /sync so the server can compute honest
  end-to-end execution_time independent of CLI-side wall clock.
- Add coverage for the argv-peek invariant: root options that take a
  value must be registered so argv-peek skips past their values.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Ilanlido Ilanlido force-pushed the CM-64735-ai-guardrails-scan-perf branch from 162847e to c2c0811 Compare June 8, 2026 12:18
@Ilanlido Ilanlido enabled auto-merge (squash) June 8, 2026 12:20
@Ilanlido Ilanlido disabled auto-merge June 8, 2026 12:20
@Ilanlido Ilanlido enabled auto-merge (squash) June 8, 2026 12:20
@Ilanlido Ilanlido merged commit 67aab55 into main Jun 8, 2026
28 checks passed
@Ilanlido Ilanlido deleted the CM-64735-ai-guardrails-scan-perf branch June 8, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants