fix: preserve torch_dtype in get_engine_config and add it to CodeFormulaV2 (#3117)
* fix: preserve torch_dtype in get_engine_config and add it to CodeFormulaV2 get_engine_config() was silently dropping torch_dtype from engine overrides when constructing EngineModelConfig. This caused CodeFormulaV2 to load in fp32 while using Flash Attention 2, which only supports fp16/bf16. Also adds torch_dtype="bfloat16" to the CodeFormulaV2 preset's TRANSFORMERS engine override. Fixes #3026 Signed-off-by: majiayu000 <1835304752@qq.com> * fix: move torch_dtype into extra_config per review feedback Signed-off-by: majiayu000 <1835304752@qq.com> --------- Signed-off-by: majiayu000 <1835304752@qq.com>
L
lif committed
53a5f80a43849d853d4e0598d3875e6aac2f88e0
Parent: 1e841eb
Committed by GitHub <noreply@github.com>
on 3/17/2026, 8:59:09 AM