mudler / LocalAI UNCLAIMED

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

0 0 66 Go

feat(llama.cpp): expose cache_type_k and cache_type_v for quant of kv cache (#4329)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

Ettore Di Giacinto committed 1y ago

d4c1746c7db3d13ba97bb9d8a8b698d8a366a0a7

Parent: 88737e1

Committed by GitHub <noreply@github.com> on 12/6/2024, 9:23:59 AM