Run example llama2 model with fp16 (#1902)
Summary: Pull Request resolved: https://github.com/pytorch/executorch/pull/1902 FYI - there are hardcoded `float` in rmsnorm which makes bunch of nodes in the graph as fp32. ``` aten_embedding_default: "f16[1, 3, 64]" = executorch_exir_dialects_edge__ops_aten_embedding_default(arg11_1, arg55_1); arg11_1 = arg55_1 = None aten_slice_copy_tensor: "f16[3, 4]" = executorch_exir_dialects_edge__ops_aten_slice_copy_Tensor(arg48_1, 0, 0, 3); arg48_1 = None aten_slice_copy_tensor_1: "f16[3, 4]" = executorch_exir_dialects_edge__ops_aten_slice_copy_Tensor(arg49_1, 0, 0, 3); arg49_1 = None aten__to_copy_default: "f32[1, 3, 64]" = executorch_exir_dialects_edge__ops_aten__to_copy_default(aten_embedding_default, dtype = torch.float32) (a lot of nodes in fp32 after this, and then we go back to fp16 and so on) ``` Copy op from - https://www.internalfb.com/code/fbsource/%5B7e45e7bcd969%5D/xplat/executorch/examples/models/llama2/model.py?lines=78 Reviewed By: larryliu0820 Differential Revision: D53596500 fbshipit-source-id: b6b3ebddfb9a25d1e52e9202d216e9ead9a6c62d
D
Digant Desai committed
eb50c4648cf40bf8cca13faecd7693fccfff02c6
Parent: 636f9a7
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 2/9/2024, 9:43:35 PM