fix(containerprofile): reduce syscall loss window on container termination (#924)
* fix(containerprofile): reduce syscall loss window on container termination The seccomp syscall tracer only fetches the advise_seccomp eBPF map on a fixed 30s poll; a container terminating between polls loses whatever it executed since the last fetch once its profile is saved and its data is removed. Make the poll interval configurable (syscallPollInterval, default 2s, down from a hardcoded 30s) and wait one bounded poll interval before the final forced profile save on termination, so the tracer's next poll cycle has a chance to land first. Fixes #922 Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * fix(containerprofile): flush syscalls on demand instead of shrinking the poll interval Bumps the inspektor-gadget fork to matthyx/inspektor-gadget#12, which adds ebpfoperator.TriggerManualMapFetch: an on-demand trigger for the eBPF map iterator that the periodic syscall poll relies on. ContainerProfileManager now calls this (via SyscallTracer.Peek, wired through the new SetSyscallFlusher hook) right before a container's final forced profile save on termination, instead of guessing how long to wait for the next scheduled poll. This replaces the previous mitigation (shrinking syscallPollInterval to 2s and sleeping up to that long on every termination) with an actual fix: the poll interval reverts to its original 30s default since it no longer needs to be short for correctness, and the wait before saving is now a fixed 500ms settle delay for the event pipeline, only paid when a flush was actually requested. Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * fix(containerprofile): drive all syscall map fetches through Peek, restore 5s alerting cadence The gadget's own internal fetch schedule is now disabled entirely (map-fetch-interval/count both "0"); every fetch, periodic or not, goes through SyscallTracer.Peek. A single background loop in Start calls Peek on syscallPollInterval (default 5s, was 30s) for live event delivery to every consumer (profile-building and RuleManager's real-time syscall alerting alike), and ContainerProfileManager calls Peek once more, out of that schedule, right before a container's final forced save. This restores the pre-PR-#591 architecture's actual property that mattered: RuleManager's real-time alerting ran on its own dedicated schedule (a 5s ticker per container) fully decoupled from ContainerProfileManager's save-driven Peek() calls. The destructive batch read the current OCI gadget uses was never actually why a single shared 30s schedule was needed; there was simply one internal ticker serving every consumer at the same cadence. Driving everything through one explicit Peek call, and making that call on the original 5s cadence, is now also cheaper than the pre-#591 design: one global fetch drains and dispatches every currently traced container in a batch, rather than one ticker per container. Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * feat(containerprofile): batch syscall reporting, bypass generic pipeline SyscallTracer.callback decodes one eBPF fetch into a batch of syscall names per container, but was still emitting one SyscallEventType event per syscall through the generic per-event pipeline (AddEventDirect, dedup-key computation, handler dispatch) for a data source that isn't actually a discrete event stream - it's a periodic snapshot of a persistent per-mntns bitmap. That per-item overhead scales with how many distinct syscalls a container has executed since its last fetch, worst right after a container starts. RuleManager's rule matching and the generic dedup cache both key off a single event.syscall value, so they still need one event per syscall. ContainerProfileManager has no such constraint - it only folds each syscall into a per-container set - so it now gets the whole batch directly via the new ReportSyscalls(containerID, []string) method, bypassing the queue/dedup/dispatch pipeline entirely for this consumer. RuleManager and metrics are unaffected, still receiving one event per syscall exactly as before. Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * fix(containerprofile): address review — flusher race, blocking dispatch, empty containerID Fixes three issues from review on PR #924 (pullrequestreview-5027546778) plus the CI failures they caused: - ContainerProfileManager.syscallFlusher was a plain func() field written once by SetSyscallFlusher (from TracerFactory, during startup wiring) and read by flushAndSettle from concurrently-running per-container monitorContainer goroutines - an unsynchronized data race that could silently skip the flush at exactly the highest-container-churn moment there is (startup/restart). Now stored via atomic.Pointer[func()]. - SyscallTracer.callback called ContainerProfileManager.ReportSyscalls synchronously and inline, on the single goroutine the gadget uses to process every currently-traced container's fetch results. ReportSyscalls can itself block (withContainer sends on a container's bounded SyncChannel while holding its lock, on a profile-size-split). Previously this same call went through the generic queue and one of many pooled workers, so one slow container never affected any other. Now dispatched on its own goroutine so it can't stall the shared fetch-processing path. - The seccomp map covers every mount namespace on the node, including ones never resolved to a container (host processes, etc). The generic pipeline used to drop those via EventHandlerFactory.ProcessEvent's empty-ContainerID check before they reached ReportSyscall; the direct call bypasses that check, so every such row was logging "invalid empty containerID" and wasting a container-map lookup, every tick - visible as hundreds of these errors in the failing CI runs (Test_02_AllAlertsFromMaliciousApp, Test_23_RuleCooldownTest, Test_28_UserDefinedNetworkNeighborhood). callback now filters these out itself before decoding or reporting. Also: shared config.DefaultSyscallPollInterval constant instead of duplicating the 5s literal in both packages, and decodeSyscalls pre-sizes its slice since the upper bound is already known. Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> * docs: correct flushAndSettle description to match the direct-call path Leftover from before the review-fix commit: ContainerProfileManager no longer goes through the generic event queue/worker pool for syscalls, so the 500ms wait description was stale (CodeRabbit review comment on PR #924). Docs-exempt: documentation-only correction, no behavioral change Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com> --------- Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>
M
Matthias Bertschy committed
0a856a2cae4b5fbff315f15dad66563dbde88e45
Parent: 6fb4444
Committed by GitHub <noreply@github.com>
on 8/26/2026, 8:14:27 AM