mudler / LocalAI UNCLAIMED

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

0 0 71 Go

chore(model-gallery): add more quants for popular models (#3365)

* models(gallery): add higher quants for some llama and hermes

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* models(gallery): vllm: specify a reasonable max_tokens

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

Ettore Di Giacinto committed 1y ago

84d6e5a9879313bfdc32de013617d7a27a03ef71

Parent: ac5f6f2

Committed by GitHub <noreply@github.com> on 8/23/2024, 10:29:24 PM