fix: synchronous endpoint + Cloud Run scaling (fixes prod timeout) (#281)
* feat: add Firestore-backed job status store Introduces FirestoreJobStore to replace the in-memory job_status_store dict, enabling any Cloud Run instance to read/write job status for multi-instance GPU scaling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add GCS output store for cross-instance file serving Implements GCSOutputStore to upload separation results to GCS so any Cloud Run instance can serve download requests, replacing local disk storage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Make /separate endpoint async with GPU semaphore and external stores Convert the /separate endpoint from synchronous (await) to fire-and-forget pattern so it returns immediately while separation runs in background. Replace in-memory job_status_store with Firestore reads/writes and local file downloads with GCS, enabling cross-instance status polling and file serving. Add GPU semaphore to serialize concurrent separation requests on a single instance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * cleanup: remove dead job_status_store dict Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: reduce POST timeout from 300s to 60s (server is now async) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add google-cloud-firestore to Dockerfile for async job store Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: lazy imports + add GCP test deps for CI compatibility Move google.cloud imports inside __init__ methods so modules can be imported without the packages installed. Add google-cloud-firestore and google-cloud-storage to dev dependencies so CI has them available. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update poetry.lock for GCP test dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip deploy_cloudrun tests when uvicorn not installed The TestLazyInit tests import deploy_cloudrun.py which requires uvicorn and fastapi - server-only dependencies not in the test environment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: synchronous endpoint + no semaphore, let Cloud Run scale like Modal The fire-and-forget + semaphore design caused all jobs to queue on one instance. Cloud Run couldn't see background threads as "busy" so it never scaled to new instances. Fix: make endpoint synchronous (await executor) with concurrency=1. Cloud Run now sees each request as active during processing and scales to new GPU instances for concurrent jobs — matching Modal's .spawn() pattern. Increase client POST timeout to 1800s to match. Keep Firestore job store and GCS output store for observability and cross-instance file access. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
A
Andrew Beveridge ☄️ committed
0a644dbb07d94a7e747d5265e3abbaf600459449
Parent: b8ded33
Committed by GitHub <noreply@github.com>
on 3/26/2026, 4:42:01 PM
No diff available for this commit.