SIGN IN SIGN UP

Add off-graph KV-cache SequenceCache and flat policy (#21412)

**Summary**

Adds the neutral single-sequence cache: a SequenceCache controller that
owns one logical length for the whole model and produces per-layer,
integer-only layout plans, plus its full-history FlatPolicy. Backend and
tensor-free the cache logic stays ET-independent, with a thin adapter
for ExecuTorch callers. The SeqStepPlan and config shapes are already
general for sliding-window policies, but only full-history flat is
implemented here.

  **Files**
- extension/llm/cache/cache.h — adds the two neutral faces recovered
from CacheBase without RTTI (SequenceControl runner face;
SequencePlanner backend face), the SeqStepPlan planner→backend handoff,
the LayoutPolicy interface, and per-layer config (LayerPolicy,
LayerConfig, CacheConfig.layers).
- extension/llm/cache/sequence_cache.h — the SequenceCache controller
(single logical length, admission, rewind,
per-layer plan dispatch with policy dedup) and FlatPolicy (full-history
layout). Header-only, ET-independent.
- extension/llm/cache/cache_et.h — thin ExecuTorch adapter mapping the
core's bool/std::optional results to
Error/Result (et::plan → OutOfResources, et::rewind → InvalidArgument).
- extension/llm/cache/cache_registry.h / cache_registry.cpp —
CacheSession gains control() (the typed runner face);
builder registry now keyed by std::pair<backend_id, kind> instead of a
concatenated string.
- extension/llm/cache/test/cache_test.cpp — tests for flat plan
(append/read-all-history), admission/rewind, face
  recovery, the registry with a real SequenceCache, and the ET adapter.

  **Testing**

Added C++ unit tests covering the flat bookkeeping
(plan/can_extend/rewind/clear), face recovery, the registry with
a real cache, and the ET adapter's result/code mapping. Built and ran
through the standard cmake/ctest flow, all
  pass:

cmake -B cmake-out -DEXECUTORCH_BUILD_EXTENSION_LLM=ON
-DEXECUTORCH_BUILD_TESTS=ON
  cmake --build cmake-out --target extension_llm_cache_test -j
  ctest --test-dir cmake-out -R extension_llm_cache --output-on-failure
K
Kiymet Akdemir committed
32f56beb724753e6690790164943695368419407
Parent: 2a64e2d
Committed by GitHub <noreply@github.com> on 7/28/2026, 10:29:50 PM