Recover containers from stuck FUSE mounts (#1824)
<!-- This is an auto-generated description by cubic. -->
## Summary by cubic
Recover containers that hang on shutdown due to stuck FUSE mounts. Adds
thread-safe lifecycle state and guarded stop escalation to safely abort
stuck mounts and prevent indefinite shutdowns.
- **Bug Fixes**
- `pkg/storage/geese.go`: On geesefs flush timeout, locate the FUSE
connection via `/proc/self/mountinfo` and abort it by writing to
`/sys/fs/fuse/connections/<id>/abort` (mounts `fusectl` if needed).
- `pkg/worker/worker.go`: Guard single stop escalation with
`StopEscalationStarted`; default `StopReason` to unknown when missing.
After SIGKILL, wait 2m; if still running on Geese storage, unmount the
workspace to recover a stuck FUSE mount, but only when
`workspaceOnlyStopping` confirms all siblings in the workspace are
stopping. Add thread-safe lifecycle state (`stateMu`, `lifecycleState`,
`setExitCode`, `setStopReason`) to avoid races.
- `pkg/worker/{lifecycle.go,criu.go,sandbox.go}`: Read/write stop
reasons and exit codes via `lifecycleState` and setters to prevent data
races. `pkg/worker/worker_test.go`: Added a concurrency test to verify
`workspaceOnlyStopping` protects running siblings.
<sup>Written for commit a5b88273eac5bf2a778c58fef3a81925b84a7a0f.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/beam-cloud/beta9/pull/1824?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. --> L
luke-lombardi committed
49dfd645890867fe0751676fd75c7412bc9a8062
Parent: acce8df
Committed by GitHub <noreply@github.com>
on 8/3/2026, 1:17:22 AM