SIGN IN SIGN UP

fix: synchronous endpoint + Cloud Run scaling (fixes prod timeout) (#281)

* feat: add Firestore-backed job status store

Introduces FirestoreJobStore to replace the in-memory job_status_store dict,
enabling any Cloud Run instance to read/write job status for multi-instance GPU scaling.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add GCS output store for cross-instance file serving

Implements GCSOutputStore to upload separation results to GCS so any
Cloud Run instance can serve download requests, replacing local disk storage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Make /separate endpoint async with GPU semaphore and external stores

Convert the /separate endpoint from synchronous (await) to fire-and-forget
pattern so it returns immediately while separation runs in background.
Replace in-memory job_status_store with Firestore reads/writes and local
file downloads with GCS, enabling cross-instance status polling and file
serving. Add GPU semaphore to serialize concurrent separation requests on
a single instance.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* cleanup: remove dead job_status_store dict

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: reduce POST timeout from 300s to 60s (server is now async)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add google-cloud-firestore to Dockerfile for async job store

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: lazy imports + add GCP test deps for CI compatibility

Move google.cloud imports inside __init__ methods so modules can be
imported without the packages installed. Add google-cloud-firestore and
google-cloud-storage to dev dependencies so CI has them available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update poetry.lock for GCP test dependencies

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip deploy_cloudrun tests when uvicorn not installed

The TestLazyInit tests import deploy_cloudrun.py which requires uvicorn
and fastapi - server-only dependencies not in the test environment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: synchronous endpoint + no semaphore, let Cloud Run scale like Modal

The fire-and-forget + semaphore design caused all jobs to queue on one
instance. Cloud Run couldn't see background threads as "busy" so it
never scaled to new instances.

Fix: make endpoint synchronous (await executor) with concurrency=1.
Cloud Run now sees each request as active during processing and scales
to new GPU instances for concurrent jobs — matching Modal's .spawn()
pattern. Increase client POST timeout to 1800s to match.

Keep Firestore job store and GCS output store for observability and
cross-instance file access.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
A
Andrew Beveridge ☄️ committed
0a644dbb07d94a7e747d5265e3abbaf600459449
Parent: b8ded33
Committed by GitHub <noreply@github.com> on 3/26/2026, 4:42:01 PM
No diff available for this commit.