SIGN IN SIGN UP

relax check on bf16 (#3460)

Summary:
Now require kernel support for bf16.

Export:
python3 torchchat.py export llama3 --dtype [fp16 | bf16] --output-pte-path llama3.pte
```
(.venv) (base) [lfq@devvm20128.prn0 /data/users/lfq/torchchat (lfq.export-bf16)]$ python3 torchchat.py generate llama3 --device cpu --pte-path llama3.pte --prompt "Hello my name is"
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Using device=cpu Intel Core Processor (Broadwell)
Loading model...
Time to load model: 0.11 seconds
I 00:00:00.000905 executorch:program.cpp:129] InternalConsistency verification requested but not available
E 00:00:51.744419 executorch:method.cpp:936] Overriding output data pointer allocated by memory plan is not allowed.
I 00:00:51.744460 executorch:pybindings.cpp:196] Cannot set_output_data_ptr(): this likely means the outputs were MemoryPlanned inspect the error code to know for sure, but likely this is not an issue. 0x2
F 00:00:51.747880 executorch:op_index.cpp:87] In function operator()(), assert failed (false): Unhandled dtype BFloat16 for index.Tensor_out
Aborted (core dumped)
```

fp16 (mostly) runs well!
```
(.venv) (base) [lfq@devvm20128.prn0 /data/users/lfq/torchchat (lfq.export-bf16)]$ python3 torchchat.py generate llama3 --device cpu --pte-path llama3_f16.pte --prompt "Hello my name is" 2>/dev/null
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Using device=cpu Intel Core Processor (Broadwell)
Loading model...
Time to load model: 0.10 seconds
Hello my name is Elsie, I'm a 28 year old woman who loves to be outdoors and try new things. I'm a bit of a thrill-seeker and love to push myself out of my comfort zone. I'm also a big fan of nature and love spending time in the wilderness.
I'm a bit of a foodSerializedName and love trying new recipes and experimenting with different flavors and ingredients. I'm also a big fan of coffee and love a good cup of joe in the morning.
I'm a bit of a homebody and love spending time at home with my family and friends. I'm also a big fan of movies and TV shows and love getting lost in a good story.
 Injector

I'm a bit of a hopeless romantic and love the idea of finding true love and building a life with someone. I'm looking for someone who is kind, honest, and genuine, and who also loves the outdoors and trying new things. If you're a fellow thrill-seeker who loves nature and<|begin_of_text|>Hello my name is Elsie, I'm a 28 year old woman who loves to be outdoors and try new things. I'm a bit of a thrill-seeker and love to push myself out of my comfort zone. I'm also a big fan of nature and love spending time in the wilderness.
I'm a bit of a foodSerializedName and love trying new recipes and experimenting with different flavors and ingredients. I'm also a big fan of coffee and love a good cup of joe in the morning.
I'm a bit of a homebody and love spending time at home with my family and friends. I'm also a big fan of movies and TV shows and love getting lost in a good story.
 Injector

I'm a bit of a hopeless romantic and love the idea of finding true love and building a life with someone. I'm looking for someone who is kind, honest, and genuine, and who also loves the outdoors and trying new things. If you're a fellow thrill-seeker who loves nature and<|eot_id|>
Max Sequence Length Reached. Ending Conversation.
==========

```

duplication seems to be resolved in https://github.com/pytorch/torchchat/pull/620

Pull Request resolved: https://github.com/pytorch/executorch/pull/3460

Reviewed By: iseeyuan

Differential Revision: D56849464

Pulled By: lucylq

fbshipit-source-id: 9cb029d55b144c1cc34ea5672ee24d3ea80d060c
L
lucylq committed
0909c5ac58b67d3f902fc888b3449f520a50e23c
Parent: 3a2b2e8
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 5/3/2024, 11:15:31 PM