Add off-graph KV-cache registry (#21383)
Summary Adds a shared, backend-agnostic registry for the off-graph KV cache: a small set of C++ pieces that let a runtime own a cache and look it up by key. Files - extension/llm/cache/cache.h — the opaque cache handle (CacheBase) and the CacheConfig struct. - extension/llm/cache/cache_registry.h / cache_registry.cpp — a process-global registry that stores caches by key, a per-(backend_id, kind) builder registry, unique-key generation, and a RAII session that installs on construction and erases on destruction. - extension/llm/cache/CMakeLists.txt — builds the library and pulls in the test folder (cmake). - extension/llm/cache/test/cache_test.cpp — unit tests for the above. - extension/llm/cache/test/CMakeLists.txt — builds the test (cmake). - CMakeLists.txt — includes the new folder in the build. Testing Added C++ unit tests (test/cache_test.cpp) covering the registry (install/get/erase, unique keys), the builder registry (build + missing-kind NotFound error), and the session's install-on-create / erase-on-destroy. Built and ran through the standard cmake/ctest flow, all pass: cmake -B cmake-out -DEXECUTORCH_BUILD_EXTENSION_LLM=ON -DEXECUTORCH_BUILD_TESTS=ON cmake --build cmake-out --target extension_llm_cache_test -j ctest --test-dir cmake-out -R extension_llm_cache --output-on-failure
K
Kiymet Akdemir committed
739fe55e5da89932a5e611f237aeb1773915f6f9
Parent: 1f0b2dc
Committed by GitHub <noreply@github.com>
on 7/27/2026, 9:58:48 PM