SIGN IN SIGN UP
mudler / LocalAI UNCLAIMED

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

0 0 66 Go

docs: Document GPU auto-fit mode limitations and trade-offs (closes #8562) (#8954)

* docs: Add documentation about GPU auto-fit mode limitations (closes #8562)

- Document the default gpu_layers behavior (9999999) that disables auto-fit
- Explain the trade-off between auto-fit and VRAM threshold unloading
- Add recommendations for users who want to enable gpu_layers: -1
- Note known issues with tensor_buft_override buffer errors
- Link to issue #8562 for future improvements

Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>

* Apply suggestion from @mudler

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

---------

Signed-off-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: team-coding-agent-1 <team-coding-agent-1@localai.dev>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
L
LocalAI [bot] committed
f73a1581539c2d6b651de4453aa9ddbff2425d54
Parent: b24ca51
Committed by GitHub <noreply@github.com> on 3/12/2026, 12:35:31 PM