SIGN IN SIGN UP

fix: preserve torch_dtype in get_engine_config and add it to CodeFormulaV2 (#3117)

* fix: preserve torch_dtype in get_engine_config and add it to CodeFormulaV2

get_engine_config() was silently dropping torch_dtype from engine
overrides when constructing EngineModelConfig. This caused
CodeFormulaV2 to load in fp32 while using Flash Attention 2, which
only supports fp16/bf16.

Also adds torch_dtype="bfloat16" to the CodeFormulaV2 preset's
TRANSFORMERS engine override.

Fixes #3026

Signed-off-by: majiayu000 <1835304752@qq.com>

* fix: move torch_dtype into extra_config per review feedback

Signed-off-by: majiayu000 <1835304752@qq.com>

---------

Signed-off-by: majiayu000 <1835304752@qq.com>
L
lif committed
53a5f80a43849d853d4e0598d3875e6aac2f88e0
Parent: 1e841eb
Committed by GitHub <noreply@github.com> on 3/17/2026, 8:59:09 AM