SIGN IN SIGN UP

integrate coreml delegate to llama_main (#4160)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/4160

As title,
build executorch library
```
cmake -DCMAKE_INSTALL_PREFIX=cmake-out \
    -DCMAKE_BUILD_TYPE=Debug \
    -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
    -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
    -DEXECUTORCH_BUILD_KERNELS_CUSTOM="$CUSTOM" \
    -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
    -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
    -DEXECUTORCH_BUILD_XNNPACK=OFF \
    -DEXECUTORCH_BUILD_COREML=ON \
    -DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
    -Bcmake-out .

cmake --build cmake-out -j16 --target install --config DEBUG
```
build llama_main binary
```
cmake -DCMAKE_INSTALL_PREFIX=cmake-out \
    -DCMAKE_BUILD_TYPE=Debug \
    -DEXECUTORCH_BUILD_KERNELS_CUSTOM="$CUSTOM" \
    -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
    -DEXECUTORCH_BUILD_XNNPACK="$XNNPACK" \
    -DPYTHON_EXECUTABLE="$PYTHON_EXECUTABLE" \
    -DEXECUTORCH_BUILD_COREML=ON \
    -Bcmake-out/examples/models/llama2 \
    examples/models/llama2

cmake --build cmake-out/examples/models/llama2 -j9 --config DEBUG
```
Run `llama_main` binary on mac
```
(executorch) chenlai@chenlai-mbp executorch % ./cmake-out/examples/models/llama2/llama_main --model_path coreml_llama2.pte --tokenizer_path /Users/chenlai/Documents/stories110M/tokenizer.bin
I 00:00:00.028804 executorch:cpuinfo_utils.cpp:62] Reading file /sys/devices/soc0/image_version
I 00:00:00.028844 executorch:cpuinfo_utils.cpp:78] Failed to open midr file /sys/devices/soc0/image_version
I 00:00:00.028847 executorch:cpuinfo_utils.cpp:158] Number of efficient cores 4
I 00:00:00.028849 executorch:main.cpp:65] Resetting threadpool with num threads = 6
I 00:00:00.033847 executorch:runner.cpp:53] Creating LLaMa runner: model_path=coreml_llama2.pte, tokenizer_path=/Users/chenlai/Documents/stories110M/tokenizer.bin
I 00:00:00.117320 executorch:runner.cpp:77] Reading metadata from model
I 00:00:00.117374 executorch:runner.cpp:130] get_n_bos: 1
I 00:00:00.117386 executorch:runner.cpp:130] get_n_eos: 1
I 00:00:00.117392 executorch:runner.cpp:130] get_max_seq_len: 128
I 00:00:00.117399 executorch:runner.cpp:130] use_kv_cache: 1
I 00:00:00.117407 executorch:runner.cpp:130] use_sdpa_with_kv_cache: 0
I 00:00:00.117412 executorch:runner.cpp:130] append_eos_to_prompt: 0
I 00:00:00.117413 executorch:runner.cpp:128] The model does not contain enable_dynamic_shape method, using default value 0
I 00:00:00.117415 executorch:runner.cpp:130] enable_dynamic_shape: 0
I 00:00:00.128599 executorch:runner.cpp:130] get_vocab_size: 512
I 00:00:00.128615 executorch:runner.cpp:130] get_bos_id: 1
I 00:00:00.128621 executorch:runner.cpp:130] get_eos_id: 2

PyTorchObserver {"prompt_tokens":9,"generated_tokens":118,"model_load_start_ms":1720221254434,"model_load_end_ms":1720221254528,"inference_start_ms":1720221254528,"inference_end_ms":1720221254817,"prompt_eval_end_ms":1720221254563,"first_token_ms":1720221254563,"aggregate_sampling_time_ms":9,"SCALING_FACTOR_UNITS_PER_SECOND":1000}
I 00:00:00.417257 executorch:runner.cpp:509] 	Prompt Tokens: 9    Generated Tokens: 118
I 00:00:00.417260 executorch:runner.cpp:515] 	Model Load Time:		0.094000 (seconds)
I 00:00:00.417266 executorch:runner.cpp:525] 	Total inference time:		0.289000 (seconds)		 Rate: 	408.304498 (tokens/second)
I 00:00:00.417268 executorch:runner.cpp:533] 		Prompt evaluation:	0.035000 (seconds)		 Rate: 	257.142857 (tokens/second)
I 00:00:00.417270 executorch:runner.cpp:544] 		Generated 118 tokens:	0.254000 (seconds)		 Rate: 	464.566929 (tokens/second)
I 00:00:00.417272 executorch:runner.cpp:552] 	Time to first generated token:	0.035000 (seconds)
I 00:00:00.417274 executorch:runner.cpp:559] 	Sampling time over 127 tokens:	0.009000 (seconds)
```
ghstack-source-id: 232863706

Reviewed By: kirklandsign

Differential Revision: D59412309

fbshipit-source-id: a31ac59e616d7333323a5c7961b8b7dafe2d45e5
C
Chen Lai committed
b91c20bc5a5cd974176230c4259474940d59e2cf
Parent: e4eeadc
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 7/8/2024, 6:25:52 PM