SIGN IN SIGN UP

fix: auto-disable batch API for providers that return 404 (#476)

## Summary

When an upstream doesn't implement the Anthropic/OpenAI batch endpoint
(e.g. MiniMax proxying the Anthropic API), every 30s flush cycle POSTs
to the batch endpoint, gets a 404, and falls back to synchronous —
wasting a round-trip on every cycle, forever. Onur reported this with
MiniMax: `batch flush → 404 → batch fallback: processing synchronously
(5/5 in the window)`.

## Changes

**`packages/gateway/src/batch-queue.ts`**:

- Extended `BatchProvider.submit()` return type to include `"not-found"`
- Added 404 handling in Anthropic submit (line 219), OpenAI file upload
(line 357), and OpenAI batch create (line 519) — all return
`"not-found"` instead of falling through to the transient `null` path
- Added 405 handling alongside 404 for OpenAI (vLLM, local models return
405 for unsupported endpoints)
- Added `disabledBatchProviders` Set (persisted to `kv_meta` via
`setKV`) — provider-level disable distinct from existing per-session
disable
- Provider-level bypass in `flush()`: skips batch submit entirely for
disabled providers
- Provider-level fast-path in `prompt()`: skips the 30s queue wait
entirely for disabled providers
- Cleaned up pre-existing duplicate 404/405 handlers in OpenAI submit
that returned `"auth-error"` (incorrect — 404 is not an auth error)

## Design: Per-Session vs Per-Provider Disable

| Signal | Scope | Rationale |
|---|---|---|
| 403 (OAuth scope) | Per-session | One session's token may lack batch
scope while another's has it |
| 404/405 (endpoint missing) | Per-provider | The upstream URL doesn't
implement batches at all — affects everyone |

Both are persisted to `kv_meta` and survive process restarts.
B
Burak Yigit Kaya committed
0684cd783cdb50670b89ce06d8775caf89627364
Parent: ac3d3cf
Committed by GitHub <noreply@github.com> on 5/27/2026, 2:15:25 PM