SIGN IN SIGN UP

Add off-graph KV-cache registry (#21383)

Summary

Adds a shared, backend-agnostic registry for the off-graph KV cache: a
small set of C++ pieces that let a runtime own a cache and look it up by
key.

  Files
- extension/llm/cache/cache.h — the opaque cache handle (CacheBase) and
the CacheConfig struct.
- extension/llm/cache/cache_registry.h / cache_registry.cpp — a
process-global registry that stores caches by key, a
per-(backend_id, kind) builder registry, unique-key generation, and a
RAII session that installs on construction
  and erases on destruction.
- extension/llm/cache/CMakeLists.txt — builds the library and pulls in
the test folder (cmake).
  - extension/llm/cache/test/cache_test.cpp — unit tests for the above.
  - extension/llm/cache/test/CMakeLists.txt — builds the test (cmake).
  - CMakeLists.txt — includes the new folder in the build.

  Testing

Added C++ unit tests (test/cache_test.cpp) covering the registry
(install/get/erase, unique keys), the builder
registry (build + missing-kind NotFound error), and the session's
install-on-create / erase-on-destroy. Built and
  ran through the standard cmake/ctest flow, all pass:

cmake -B cmake-out -DEXECUTORCH_BUILD_EXTENSION_LLM=ON
-DEXECUTORCH_BUILD_TESTS=ON
  cmake --build cmake-out --target extension_llm_cache_test -j
  ctest --test-dir cmake-out -R extension_llm_cache --output-on-failure
K
Kiymet Akdemir committed
739fe55e5da89932a5e611f237aeb1773915f6f9
Parent: 1f0b2dc
Committed by GitHub <noreply@github.com> on 7/27/2026, 9:58:48 PM