SIGN IN SIGN UP
mudler / LocalAI UNCLAIMED

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

0 0 102 Go

fix: drop gguf VRAM estimation (now redundant) (#8325)

fix: drop gguf VRAM estimation

Cleanup. This is now handled directly in llama.cpp, no need to estimate from Go.

VRAM estimation in general is tricky, but llama.cpp ( https://github.com/ggml-org/llama.cpp/blob/41ea26144e55d23f37bb765f88c07588d786567f/src/llama.cpp#L168 ) lately has added an automatic "fitting" of models to VRAM, so we can drop backend-specific GGUF VRAM estimation from our code instead of trying to guess as we already enable it

 https://github.com/mudler/LocalAI/blob/397f7f0862d4105b874523e1a0105ae036db18ec/backend/cpp/llama-cpp/grpc-server.cpp#L393

Fixes: https://github.com/mudler/LocalAI/issues/8302
See: https://github.com/mudler/LocalAI/issues/8302#issuecomment-3830773472
E
Ettore Di Giacinto committed
800f749c7ba2fdefce6270fc22167a256a418983
Parent: b6459dd
Committed by GitHub <noreply@github.com> on 2/1/2026, 4:33:28 PM