Bugfix: Remove f16_kv, add offload_kqv field (#1019)
F16_KV appears to have been removed here: https://github.com/ggerganov/llama.cpp/pull/4312/commits/af99c6fbfc815df7dad94d8c1f20d55927b2203a This addresses two issues: - #995 which just requests to add the KV cache offloading param - #1006 a NULL ptr exception when using the embeddings (introduced by leaving f16_kv in the fields struct)
B
Brandon Roberts committed
62944df1425016018b5a137ebd0af5495eaa4894
Parent: 37da8e8
Committed by GitHub <noreply@github.com>
on 12/18/2023, 7:27:11 PM