SIGN IN SIGN UP

Pass routing headers to ChatGPT subscription API (#62556)

## Objective

Improve cache hit rate of LLM requests sent to ChatGPT subscription API.

## Solution

Set headers that help route request to the correct servers. This is
ChatGPT specific; plain OpenAI API doesn't require it.

## Testing

I run a benchmark which imitates a thread that makes 20 sequential tool
calls. I tried different cache route strategies. Results are below:


| Configuration                                   | Hit rate |
|-------------------------------------------------|---------:|
| No identity headers, no cache key               | 22.2%    |
| `thread-id` only, no cache key                  | 44.4%    |
| `thread-id` + matching cache key                | 42.1%    |
| Cache key only                                  | 53.6%    |
| Turn state + cache key                          | 47.4%    |
| `session-id` + matching cache key               | 100.0%   |
| `session-id` + `thread-id` + matching cache key | 97.4%    |



---

Release Notes:

- Improved ChatGPT subscription caching
O
Oleksiy Syvokon committed
a8fafdd7ee36fb3fb98ebbfe5d3be983301d9e74
Parent: c05e346
Committed by GitHub <noreply@github.com> on 8/13/2026, 1:04:16 AM