mudler / LocalAI UNCLAIMED

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

0 0 66 Go

fix(config): set better defaults for inferencing (#1822)

* fix(defaults): set better defaults for inferencing

This changeset aim to have better defaults and to properly detect when
no inference settings are provided with the model.

If not specified, we defaults to mirostat sampling, and offload all the
GPU layers (if a GPU is detected).

Related to https://github.com/mudler/LocalAI/issues/1373 and https://github.com/mudler/LocalAI/issues/1723

* Adapt tests

* Also pre-initialize default seed

Ettore Di Giacinto committed 2y ago

f895d066055728e2744044ce6390a222bc24d095

Parent: bc8f648

Committed by GitHub <noreply@github.com> on 3/13/2024, 9:05:30 AM