fix: synchronous endpoint + no semaphore, let Cloud Run scale like Modal (#282)
The fire-and-forget + semaphore design caused all jobs to queue on one instance. Cloud Run couldn't see background threads as "busy" so it never scaled to new instances. Fix: make endpoint synchronous (await executor) with concurrency=1. Cloud Run sees each request as active during processing and scales to new GPU instances for concurrent jobs — matching Modal's .spawn(). Increase client POST timeout to 1800s to match. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A
Andrew Beveridge ☄️ committed
b8ded3302b6ee30c1f135815711eb625938955dc
Parent: 349ba98
Committed by GitHub <noreply@github.com>
on 3/26/2026, 4:35:44 PM