feat(sandbox): log connection attempts that bypass proxy path

[pimlock]

Problem

When a process inside a sandbox attempts an outbound connection without going through the configured HTTP proxy path, we currently get poor observability.

In a recent openclaw-local debugging session, OpenClaw's Node fetch() failed with fetch failed / UND_ERR_CONNECT_TIMEOUT while curl to the same https://inference.local/... endpoint succeeded. The difference was that curl honored the sandbox proxy environment automatically, while Node fetch() required NODE_USE_ENV_PROXY=1.

Because the failing path did not produce useful OpenShell connection logs, it was hard to tell whether the app:

• never emitted traffic
• attempted a direct socket connection
• bypassed the proxy path
• was blocked before the request reached the normal logging path

John suggested that seccomp may be a good place to catch the socket attempt before we block it.

Requested change

Add sandbox-level diagnostics for outbound connection attempts that do not go through the expected proxy path.

The main goal is to surface enough information to debug cases where application networking fails before normal proxy / connection logs appear.

What to log

At minimum, when feasible, log:

• process / binary identity
• destination address and port requested by the process
• whether the connection was direct vs proxy-mediated
• reason the attempt was blocked or considered invalid
• enough correlation data to tie the event back to a sandbox and process

Possible implementation direction

Investigate whether seccomp can intercept socket / connect attempts early enough to log them before denial, especially for direct connections that bypass the proxy path.

Acceptance criteria

• A direct connection attempt from inside a sandbox that does not use the proxy path produces an observable diagnostic event
• The diagnostic includes destination and process identity
• The event is visible through existing sandbox / gateway logging surfaces, or a clearly documented new surface
• The logs make it obvious why a request like fetch failed produced no normal connection log

Repro context

Inside sandbox openclaw-local:

• curl https://inference.local/v1/models succeeded
• plain Node fetch(https://inference.local/v1/models) timed out
• NODE_USE_ENV_PROXY=1 node ...fetch(...) succeeded

This suggests we need better observability for connection attempts that never enter the expected proxy/logging path.

[johntmyers]

🏗️ build-plan

Implementation Plan

Issue type: feat
Complexity: Medium
Confidence: High — the architecture already has all necessary hooks; this is new iptables rules + a log reader task

Summary

When a sandbox process attempts a direct outbound connection without going through the HTTP CONNECT proxy (e.g., Node.js fetch() without NODE_USE_ENV_PROXY=1), the connection currently hangs silently for 30+ seconds until timeout — no log, no error, complete observability blackout. This plan adds iptables rules inside the sandbox network namespace to immediately reject bypass attempts (ECONNREFUSED instead of timeout) and a background /dev/kmsg reader to surface structured diagnostic events through existing logging infrastructure.

Architecture Analysis

Why seccomp won't work: In proxy mode, AF_INET/AF_INET6 are allowed (process needs TCP to talk to the proxy). Seccomp BPF cannot dereference the sockaddr pointer in connect() to distinguish proxy vs. direct destinations.

Why iptables LOG + REJECT is the right mechanism:

Mechanism	Verdict	Reason
Seccomp	❌ Not viable	Cannot dereference sockaddr pointer; can't distinguish proxy vs. direct connect
eBPF	❌ Over-engineered	Adds BPF loader dependency, requires CAP_BPF, massive complexity for a diagnostic feature
iptables LOG + REJECT	✅ Chosen	Uses existing CAP_NET_ADMIN, zero new capability requirements, provides both fast-fail UX AND diagnostic logging

Prerequisites

iptables must be added to the sandbox container image. It is not currently installed — only iproute2 is present. The supervisor already has all required capabilities (CAP_NET_ADMIN, root UID).

Where	What	Notes
openshell-community repo	Add iptables to sandbox base image Dockerfile	Same pattern as iproute2 — required for this feature
examples/bring-your-own-container/Dockerfile	Add iptables to package list	So BYOC users get bypass diagnostics
architecture/sandbox-custom-containers.md	Document iptables as optional dependency	Bypass diagnostics degrade gracefully without it

The feature uses the same graceful degradation pattern as iproute2: if iptables is not found at runtime, log a warning and skip rule installation. The namespace still provides isolation via routing — just without fast-fail and diagnostics.

Scope

• crates/openshell-sandbox/src/sandbox/linux/netns.rs: Add iptables rules during namespace creation; run_iptables_netns() helper
• crates/openshell-sandbox/src/bypass_monitor.rs (NEW): /dev/kmsg reader task, LOG line parser, process identity resolution, structured event emission
• crates/openshell-sandbox/src/lib.rs: Register bypass_monitor module; spawn monitor task after netns creation; wire up denial_tx
• examples/bring-your-own-container/Dockerfile: Add iptables to package list
• architecture/sandbox.md: Document bypass detection subsystem
• architecture/sandbox-custom-containers.md: Document iptables as optional dependency for bypass diagnostics

Files NOT changed:

• seccomp.rs — cannot distinguish proxy vs. direct (same socket domain)
• proxy.rs — never sees bypass traffic
• openshell-server/ — bypass events flow through existing LogPushLayer and DenialAggregator paths
• openshell-policy/ — no policy schema changes

External dependency (separate PR to openshell-community repo):

• Sandbox base image Dockerfile — add iptables package

Implementation Steps

Step 1: Add iptables rules to sandbox network namespace
File: crates/openshell-sandbox/src/sandbox/linux/netns.rs

After the default route is configured in NetworkNamespace::create(), add iptables rules inside the sandbox namespace:

OUTPUT chain — IPv4 (iptables) and IPv6 (ip6tables):
1. ACCEPT  -d 10.200.0.1/32 -p tcp --dport 3128     (proxy traffic)
2. ACCEPT  -o lo                                       (loopback)
3. ACCEPT  -m conntrack --ctstate ESTABLISHED,RELATED  (response packets)
4. LOG     -p tcp --syn --log-prefix "openshell:bypass:{id}:" --log-uid -m limit --limit 5/sec  (TCP diagnostic)
5. REJECT  -p tcp --reject-with icmp-port-unreachable   (TCP fast-fail)
6. LOG     -p udp --log-prefix "openshell:bypass:{id}:" --log-uid -m limit --limit 5/sec  (UDP diagnostic)
7. REJECT  -p udp --reject-with icmp-port-unreachable   (UDP fast-fail — covers DNS bypass)

Key design decisions:

• REJECT instead of DROP: application gets immediate ECONNREFUSED instead of a 30-second timeout
• Proxy port parameterized (not hardcoded to 3128)
• Graceful degradation: if iptables not found, log warning and skip — namespace still isolates via routing
• IPv6 rules mirrored via ip6tables
• UDP rules included to cover DNS bypass and other non-TCP protocols

Step 2: Implement /dev/kmsg bypass monitor
File: crates/openshell-sandbox/src/bypass_monitor.rs (NEW)

A background tokio task that:

1. Opens /dev/kmsg, seeks to end (skip historical messages)
2. Reads new kernel log messages via a blocking thread + mpsc channel
3. Filters for openshell:bypass:{namespace_id}: prefix
4. Parses iptables LOG format: DST=, DPT=, SPT=, UID=, PROTO=
5. Attempts process identity resolution via existing procfs::resolve_tcp_peer_identity() (best-effort, TCP only)
6. Emits structured tracing::warn!() matching existing proxy log conventions (see Example Output below)
7. Sends a DenialEvent to the denial aggregator with denial_stage: "bypass" (visibility only — no policy proposals)

Graceful degradation: if /dev/kmsg can't be opened, log a one-time warning and return.

Step 3: Wire up in sandbox lifecycle
File: crates/openshell-sandbox/src/lib.rs

After NetworkNamespace::create() succeeds and before ProcessHandle::spawn(), spawn the bypass monitor task (Linux only, gated with #[cfg(target_os = "linux")]).

Step 4: Update BYOC example and documentation

• examples/bring-your-own-container/Dockerfile: Add iptables to apt-get install line
• architecture/sandbox.md: Add "Bypass Detection" section
• architecture/sandbox-custom-containers.md: Document iptables as optional dependency

Example Output

The bypass monitor emits structured tracing events that follow the same field conventions as existing proxy log lines (CONNECT, FORWARD). The sandbox_id is injected at the transport layer by LogPushLayer — it does not appear in the tracing fields directly.

Tracing call (Rust source)

warn!(
    dst_addr = %dst_ip,
    dst_port = dst_port,
    proto = %proto,
    binary = %binary_path,
    binary_pid = %pid_str,
    ancestors = %ancestors_str,
    action = "reject",
    reason = "direct connection bypassed HTTP CONNECT proxy",
    hint = %hint_str,
    "BYPASS_DETECT",
);

Example 1: Node.js fetch() bypassing proxy (TCP)

The original issue's scenario — Node.js fetch() without NODE_USE_ENV_PROXY=1:

2026-03-15T18:30:42.123Z  WARN openshell_sandbox::bypass_monitor:
  BYPASS_DETECT
    dst_addr=93.184.216.34 dst_port=443 proto=tcp
    binary=/usr/bin/node binary_pid=4821
    ancestors=/usr/bin/bash -> /usr/bin/node
    action=reject
    reason="direct connection bypassed HTTP CONNECT proxy"
    hint="ensure process honors HTTP_PROXY/HTTPS_PROXY; for Node.js set NODE_USE_ENV_PROXY=1"

Compare with how a normal proxied connection looks:

2026-03-15T18:30:42.123Z  INFO openshell_sandbox::proxy:
  CONNECT
    src_addr=10.200.0.2 src_port=48012 proxy_addr=10.200.0.1:3128
    dst_host=inference.local dst_port=443
    binary=/usr/bin/node binary_pid=4821
    ancestors=/usr/bin/bash -> /usr/bin/node
    action=allow engine=opa policy=default
    reason=-

Example 2: DNS bypass attempt (UDP)

A process tries direct DNS resolution instead of using the proxy's resolver:

2026-03-15T18:30:43.456Z  WARN openshell_sandbox::bypass_monitor:
  BYPASS_DETECT
    dst_addr=8.8.8.8 dst_port=53 proto=udp
    binary=/usr/bin/python3 binary_pid=5102
    ancestors=/usr/bin/bash -> /usr/bin/python3
    action=reject
    reason="direct connection bypassed HTTP CONNECT proxy"
    hint="DNS queries should route through the sandbox proxy; check resolver configuration"

Example 3: Process identity unavailable (race condition)

Socket closed before /proc lookup — binary/PID fall back to "-":

2026-03-15T18:30:44.789Z  WARN openshell_sandbox::bypass_monitor:
  BYPASS_DETECT
    dst_addr=10.0.0.5 dst_port=6379 proto=tcp
    binary=- binary_pid=-
    ancestors=-
    action=reject
    reason="direct connection bypassed HTTP CONNECT proxy"
    hint="ensure process honors HTTP_PROXY/HTTPS_PROXY"

DenialEvent (fed to aggregator)

Each bypass event also produces a DenialEvent for the DenialAggregator:

DenialEvent {
    host: "93.184.216.34",       // raw IP (no DNS — bypass skipped proxy resolver)
    port: 443,
    binary: "/usr/bin/node",     // or "-" if unavailable
    ancestors: vec!["/usr/bin/bash", "/usr/bin/node"],
    deny_reason: "direct connection bypassed HTTP CONNECT proxy",
    denial_stage: "bypass",      // new stage, alongside "connect", "forward", "ssrf", "l7"
    l7_method: None,
    l7_path: None,
}

This deduplicates by (host, port, binary) and flushes via SubmitPolicyAnalysis gRPC to the gateway, same as existing denial stages.

Wire format (SandboxLogLine proto)

As seen by the gateway after LogPushLayer serializes the tracing event:

SandboxLogLine {
    sandbox_id: "openclaw-local",
    timestamp_ms: 1742063442123,
    level: "WARN",
    target: "openshell_sandbox::bypass_monitor",
    message: "BYPASS_DETECT",
    source: "sandbox",
    fields: {
        "dst_addr": "93.184.216.34",
        "dst_port": "443",
        "proto": "tcp",
        "binary": "/usr/bin/node",
        "binary_pid": "4821",
        "ancestors": "/usr/bin/bash -> /usr/bin/node",
        "action": "reject",
        "reason": "direct connection bypassed HTTP CONNECT proxy",
        "hint": "ensure process honors HTTP_PROXY/HTTPS_PROXY; for Node.js set NODE_USE_ENV_PROXY=1"
    }
}

Test Plan

• Unit tests: Parse various iptables LOG lines from /dev/kmsg — parser correctness for IPv4, IPv6, TCP, UDP, truncated, non-matching lines. Verify DenialEvent with denial_stage: "bypass" aggregates correctly.
• Integration tests: (#[ignore], requires root) Create namespace, verify iptables rules present (both TCP and UDP). Direct TCP connect inside namespace gets ECONNREFUSED. Direct UDP send gets ICMP unreachable.
• E2E tests: Run sandbox, attempt direct socket connect, check logs for bypass diagnostic event.

Risks & Open Questions

Risk	Severity	Mitigation
iptables not in sandbox image	Medium	Graceful degradation: skip with warning if not found. Add to base image in openshell-community repo.
/dev/kmsg restricted in container	Low	Graceful degradation — REJECT still works, just no diagnostic events
xt_LOG kernel module not loaded	Low	If LOG rule fails, fall back to REJECT-only; log warning
Process identity race (socket closed before reader)	Low	Accept best-effort identity; DST IP:port is always available and most useful
Log volume from aggressive retry loops	Low	iptables --limit 5/sec --limit-burst 10 rate limiting

Resolved questions:

1. Should bypass events generate policy proposals? No — bypass indicates a proxy config problem in the app, not a missing network policy. Events flow through denial aggregator for visibility only.
2. UDP bypass logging? Yes — included in initial implementation (Step 1 rules 6-7). Covers DNS bypass and other non-TCP protocols.
3. Is iptables available in the sandbox? Not currently. Needs to be added to the sandbox base image (openshell-community repo) and BYOC example. Supervisor already has CAP_NET_ADMIN and root — no capability changes needed. Feature degrades gracefully if absent.

Documentation Impact

• architecture/sandbox.md: Add bypass detection subsystem documentation
• architecture/sandbox-custom-containers.md: Document iptables as optional dependency for bypass diagnostics

---

Revision 4 — add Prerequisites section for iptables package dependency; add BYOC Dockerfile and sandbox-custom-containers.md to scope; resolve iptables availability question
Revision 3 — add example output section: tracing calls, rendered log lines (TCP/UDP/race), DenialEvent struct, SandboxLogLine wire format
Revision 2 — include UDP bypass logging in initial implementation; confirm no policy proposals from bypass events
Revision 1 — initial plan

[johntmyers]

🏗️ build-from-issue-agent

1 - agreed. no.

Confirmed — bypass events will not generate policy proposals. They'll flow through the denial aggregator for visibility only.

2 - yes, include it

Will include UDP bypass logging (LOG + REJECT) in the initial implementation rather than deferring to a follow-up. This covers DNS bypass scenarios. The iptables rules in Step 1 will be extended with matching UDP rules for both IPv4 and IPv6.

Updating the plan now.

[johntmyers]

🏗️ build-from-issue-agent

I don't see iptables available when I start a sandbox, is this just going to be something we need to make available to the sandbox supervisor?

Correct. iptables is not currently in the sandbox base image — only iproute2 is. But the supervisor already has everything else needed:

• CAP_NET_ADMIN is already granted (crates/openshell-server/src/sandbox/mod.rs:946)
• Supervisor runs as root (UID 0)

So it's the same pattern as iproute2 — a package that needs to be present in the sandbox image for the feature to activate. The plan already includes graceful degradation (check for iptables availability, skip with a warning if absent), so existing images won't break.

Concrete changes needed:

1. openshell-community repo — add iptables to the sandbox base image Dockerfile (alongside iproute2)
2. BYOC example (examples/bring-your-own-container/Dockerfile) — add iptables to the package list
3. Documentation (architecture/sandbox-custom-containers.md) — document iptables as an optional dependency for bypass diagnostics, alongside the existing iproute2 requirement

Updating the plan to include these as explicit scope items.

[johntmyers]

🏗️ build-from-issue-agent

Confirm that userland would not be able to mess w/ iptables

Confirmed — the sandbox user process cannot modify iptables rules. The supervisor sets up the rules before spawning the child, and the child has no path to modify or remove them.

Primary protection: The child process drops to the sandbox user (non-root) via setuid() (process.rs:423). The kernel clears CAP_NET_ADMIN from the effective and permitted capability sets on setuid to non-root. Without CAP_NET_ADMIN, the kernel rejects all iptables/netfilter operations regardless of the interface used (netlink sockets, setsockopt(IPT_SO_SET_REPLACE), etc.).

Defense-in-depth layers:

Layer	Mechanism	Location
Policy validation	Rejects any run_as_user other than "sandbox"	openshell-policy/src/lib.rs:470-481
no_new_privs	prctl(PR_SET_NO_NEW_PRIVS) prevents escalation via setuid binaries	seccomp.rs:25
Re-acquisition check	Verifies setuid(0) fails after privilege drop (CERT POS37-C)	process.rs:436-444
Landlock	/sys not accessible at all; /proc read-only — blocks filesystem-based netfilter attacks	landlock.rs:37-65
Namespace isolation	Child shares the netns but can't escape (no CAP_SYS_ADMIN for unshare(CLONE_NEWNET))	netns.rs:53-169

Nuance: The seccomp profile allows AF_NETLINK in Proxy mode (needed for legitimate operations). The capability check — not the socket domain — is what gates netfilter access. Blocking NETLINK_NETFILTER specifically could be a defense-in-depth follow-up but is not required for safety.

No plan changes needed — the design is sound.

[johntmyers]

🏗️ build-from-issue-agent

Implementation Complete

PR: #326

What was built

Added iptables LOG + REJECT rules inside the sandbox network namespace for bypass detection. Applications now get immediate ECONNREFUSED instead of a 30-second timeout when they bypass the proxy, and a /dev/kmsg monitor emits structured BYPASS_DETECT tracing events with destination, protocol, process identity, and actionable hints (e.g., "for Node.js set NODE_USE_ENV_PROXY=1").

Tests

• Unit: 11 tests added (kmsg parsing, field extraction, hint generation)
• Integration: N/A (iptables rule tests require root)
• E2E: N/A

Docs updated

• architecture/sandbox.md: Bypass detection section
• architecture/sandbox-custom-containers.md: iptables as optional dependency

External dependency

Base image iptables package: NVIDIA/OpenShell-Community#36 (merged)

The issue will auto-close when the PR is merged.