fix: remove noisy per-PID trace logs from fd usage collection#1246
fix: remove noisy per-PID trace logs from fd usage collection#1246purple4reina wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR reduces log noise in the /proc-based usage-metrics monitoring loop by removing per-PID trace! logging when /proc/{pid}/fd is unreadable, aligning fd collection behavior with other /proc/{pid} readers that already fail silently.
Changes:
- Removed
trace!logging onfs::read_dir("/proc/{pid}/fd")failures (benign permission/readability cases). - Dropped the unused
traceimport fromtracing. - Minor simplification: inline
files.count()into the accumulator update.
|
Related to #535 |
|
duncanista
left a comment
duncanista
left a comment
There was a problem hiding this comment.
Don't think the solution is removing it, but rather reducing how often this is logged, cc @shreyamalpani since you added this code, any opinions?
|
I believe PID 1 is the Lambda init process and we are probably seeing this failure log because of a permissions issue. I think this is safe to remove for now, but if we do start seeing file descriptor metrics drop we should add this back, ignoring any failures on PID 1 as that is a known failure. |
purple4reina
force-pushed
the
rey.abolofia/fd-logs
branch
2 times, most recently
from
c3c7925 to
be8f41d
Compare
The usage-metrics monitoring loop polls /proc every 10ms (100Hz) and reads
/proc/{pid}/fd for every PID. When a read_dir fails (e.g. /proc/1/fd is not
readable), it logged a trace line per PID per tick — ~40 messages/invocation
(~4000 per 100 invocations) when trace logging is enabled.
These failures are benign (a PID's fd dir may be unreadable while it is
otherwise alive), so drop the logging and fail silently with `continue`,
matching the other /proc/{pid} readers (fd_max, threads_max, threads_use).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
What does this PR do?
The usage-metrics monitoring loop polls
/procevery 10ms and reads/proc/{pid}/fdfor every PID each tick. Whenread_dirfails (e.g./proc/1/fdis not readable by the extension), the existing code logged atrace!message per unreadable PID per tick. At ~40 ticks per invocation (tested with a Hello World golang function), a single persistently-unreadable PID produced ~40 trace messages per invocation (~4000 per 100 invocations).These failures are benign — a PID's
fddirectory may be unreadable while the process is otherwise alive (typically a permission issue). The other/proc/{pid}readers (fd_max,threads_max,threads_use) already fail silently withcontinue. This change makesget_fd_use_data_from_pathconsistent with that behavior by removing the trace logging and dropping the now-unusedtraceimport.Motivation
Observed ~4000 noisy log messages per 100 invocations when trace logging was enabled, making trace output unusable for debugging.
Testing
cargo build— clean compile, no unused-import warningscargo test proc— all 8 proc tests pass, includingtest_get_fd_use_datacargo clippy— cleanChecklist