fix: persist terminal status on generic SBOM-generation failures to stop reprocessing loop (#855)

* fix: persist terminal status on generic SBOM-generation failures to stop reprocessing loop

SBOM-generation failures (sidecar scan error, invalid image source, syft
cataloging error) reported via reportFailure but never marked the reserved
SBOM object with a terminal status. Since the reprocessing switch only
special-cases TooLarge and Learning, a permanently-failing image fell
through to the "processing was interrupted, retrying" default case and
was silently reprocessed on every subsequent container start for that
image, producing repeated identical error logs and backend failure
reports forever.

Add markSBOMStatus, generalizing the existing TooLarge-marking pattern,
and use it to persist an Incomplete status on all three generic-failure
call sites. Add a matching Incomplete case to the reprocessing switch,
version-gated exactly like the existing Learning case, so a later
node-agent build still retries images that previously failed.

This is a pre-existing defect independent of #853/#854 (it already
affected ordinary syft SBOM-generation failures, unrelated to digests);
splitting it into its own change keeps each fix minimal and reviewable.

Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing or terminal-status behavior

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>

* fix: never overwrite existing SBOM content on a failed reprocess

Addresses review feedback: markSBOMStatus persisted a wipSbom fetched via
GetSBOMMeta, which the storage layer returns without its Spec (metadata-
only fetch). Reprocessing a previously-successful, content-bearing SBOM
(e.g. after a node-agent version bump) that then fails would silently
overwrite its real content with an empty Spec and pin it to a terminal
status -- permanently losing vulnerability-scan coverage for that image.

Track whether the SBOM being reprocessed had prior successful content
(wipSbomHadContent, set only in the Learning-case version-mismatch
branch) and skip the destructive persist whenever it did, across every
path that can reach it: the generic-failure branches (handleGenericFailure,
now with a bounded failureRetries counter so a single transient error
doesn't permanently pin an image either), the scanner-crash branch
(handleScannerCrash), and the ErrImageTooLarge branch, whose totalSize is
computed from the currently-mounted layer paths rather than being a fixed
property of the image and so is equally reachable while reprocessing.

Also extracts shouldRetryAtCurrentVersion, shared by the Learning and
Incomplete switch cases, removing the near-duplicate version-gating logic
and the comment that had drifted between them.

Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing or terminal-status behavior

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>

* fix: mark SBOM terminal status via annotation-only patch, not full replace

Addresses non-blocking review feedback: the previous fix's hadContent
guard meant a previously-successful SBOM that started failing
permanently (not just transiently) was reprocessed on every container
start forever, since nothing was ever persisted to stop the loop for
that class of image -- the retry-bounding only applied to images that
never had content.

Add storage.SbomClient.PatchSBOMAnnotations, a JSON merge patch on
metadata.annotations only, which never sends spec regardless of what
the caller does or doesn't know about the object's content. Route all
SBOM status marking (Incomplete, TooLarge) through it instead of a full
ReplaceSBOM, so it's always safe to persist a terminal status -- the
hadContent tracking, and every guard built on it across
handleGenericFailure, handleScannerCrash and the ErrImageTooLarge
branch, is removed entirely. Retries are now bounded uniformly for all
images via the same failureRetries counter, switched from a plain map
to a bounded+TTL'd expirable.LRU so short-lived images don't leak
entries either (the second non-blocking item).

markSBOMStatus now also records the current tool version alongside the
status, since the version check that gates future reprocessing depends
on it and the object is no longer implicitly carrying an in-memory
version bump the way the old ReplaceSBOM-based code did.

Docs-exempt: pure bug fix, no existing doc describes SBOM reprocessing, terminal-status, or storage patch behavior

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>

* fix: never mark a content-bearing SBOM TooLarge, a storage-layer one-way door

Addresses review feedback on d95955c7: TooLarge is special-cased in the
storage layer's GuaranteedUpdate, which silently drops every future
write (patch or replace) to an object once its status annotation is
TooLarge. Every other TooLarge writer in this codebase explicitly
clears Spec first because of this, but PatchSBOMAnnotations never
touches Spec at all -- so patching a content-bearing SBOM to TooLarge
left its real Spec permanently frozen in storage: unfixable by any
future reprocess, and defeating the point of TooLarge in the first
place (avoiding a bloated stored object).

Reintroduce wipSbomHadContent, scoped narrowly to the two TooLarge
write sites (the ErrImageTooLarge branch and handleScannerCrash's
post-maxScanRetries marking): a content-bearing SBOM now falls back to
the retryable Incomplete path instead, which has no such short-circuit.
Incomplete continues to flow entirely through the annotation-only
patch introduced in d95955c7, unaffected.

Docs-exempt: pure bug fix, no existing doc describes SBOM status transitions or storage patch semantics

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>

---------

Signed-off-by: Matthias Bertschy <matthias.bertschy@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>