Add off-graph KV-cache SequenceCache and flat policy (#21412)
**Summary** Adds the neutral single-sequence cache: a SequenceCache controller that owns one logical length for the whole model and produces per-layer, integer-only layout plans, plus its full-history FlatPolicy. Backend and tensor-free the cache logic stays ET-independent, with a thin adapter for ExecuTorch callers. The SeqStepPlan and config shapes are already general for sliding-window policies, but only full-history flat is implemented here. **Files** - extension/llm/cache/cache.h — adds the two neutral faces recovered from CacheBase without RTTI (SequenceControl runner face; SequencePlanner backend face), the SeqStepPlan planner→backend handoff, the LayoutPolicy interface, and per-layer config (LayerPolicy, LayerConfig, CacheConfig.layers). - extension/llm/cache/sequence_cache.h — the SequenceCache controller (single logical length, admission, rewind, per-layer plan dispatch with policy dedup) and FlatPolicy (full-history layout). Header-only, ET-independent. - extension/llm/cache/cache_et.h — thin ExecuTorch adapter mapping the core's bool/std::optional results to Error/Result (et::plan → OutOfResources, et::rewind → InvalidArgument). - extension/llm/cache/cache_registry.h / cache_registry.cpp — CacheSession gains control() (the typed runner face); builder registry now keyed by std::pair<backend_id, kind> instead of a concatenated string. - extension/llm/cache/test/cache_test.cpp — tests for flat plan (append/read-all-history), admission/rewind, face recovery, the registry with a real SequenceCache, and the ET adapter. **Testing** Added C++ unit tests covering the flat bookkeeping (plan/can_extend/rewind/clear), face recovery, the registry with a real cache, and the ET adapter's result/code mapping. Built and ran through the standard cmake/ctest flow, all pass: cmake -B cmake-out -DEXECUTORCH_BUILD_EXTENSION_LLM=ON -DEXECUTORCH_BUILD_TESTS=ON cmake --build cmake-out --target extension_llm_cache_test -j ctest --test-dir cmake-out -R extension_llm_cache --output-on-failure
K
Kiymet Akdemir committed
32f56beb724753e6690790164943695368419407
Parent: 2a64e2d
Committed by GitHub <noreply@github.com>
on 7/28/2026, 10:29:50 PM