feat(networkstream): bound the process trees per message, and make the size observable (SUB-7786)
Attribution changed what sizes the payload. The old batch key collapsed every process reaching one endpoint into a single entry carrying no tree, so size tracked distinct ENDPOINTS. The new key splits per process and each distinct process contributes a tree, so size tracks distinct PROCESSES that connected -- and nothing bounded that. SUB-7850's analysis modelled bytes per connection at a fixed 283 connections, which is precisely the quantity the key change stops holding fixed, so the multiplier on connection count was never budgeted. Measured with ~2 KB trees: 4,000 distinct processes produce 4.99 MB of JSON, or 6.65 MB once the synchronizer envelope's base64 applies -- over the 5 MiB limit. That is not graceful: sendNetworkEvent gets a non-2xx, Start() logs it and drops the snapshot, so the node loses its ENTIRE interval of traffic. Reachable on a node with heavy short-lived process churn. maxProcessTreeBytes budgets the trees at 1.5 MiB of estimated bytes. A byte budget rather than a tree count, because tree size still varies ~2.5x under the command-line cap. Connections are never dropped -- that is the data loss this change exists to fix -- only trees, and the refs stay put, so pid identity survives and ProcessTreeFor returns nil for them as specified. Candidates rank by connection count (highest fan-out is both the costliest attribution to lose and the shape reputation cares about), ties broken on the ref so the payload never depends on map iteration order. How often this binds in reality is NOT knowable from current data: today's sensor strips trees and emits no process identity, so distinct-processes-per-batch exists in no message. Fleet mean today is ~30 KB (pulsar_average_msg_size on network-stream-v1), ~175x under the limit -- comfortable baseline, but silent on the new multiplier. So both the budget firing and any payload above 2 MiB now log with their shape, which is what makes the question answerable after rollout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Alon <alon@armosec.io>
A
Alon committed
0dd6419850290b2eb8bb24c290a8a28394821e6e
Parent: dee5d90