Add new mps runtime with support for FP16, lifted and unlifted graphs (iOS15+, macOS12+) (#1655)
Summary: This PR changes the MPS Backend runtime to support for **iOS15+/macOS12+** (previous runtime was limited to iOS17/macOS14 only). Additionally, this PR contains changes such as support for both lifted and unlifted graphs, support for torch.export API, optimizations for FP16 and faster model loading during runtime (more information in the summary). **Summary of changes:** - Add support for running the models in FP16 (https://github.com/DenisVieriu97/executorch/pull/7 georgepaw) - Replace the previous MPS runtime from ExecuTorch which was relying on `iOS17` / `macOS` Sonoma APIs for serialization of the MPSGraphExecutable. Instead of creating the MPSGraph nodes and serializing them during AOT, create the corresponding entries of the EdgeIR nodes in the FlatBuffer, and parse them during runtime to construct the graph. This removes any dependency on iOS17 / macOS 14.0 APIs. - Add support for node visitor pattern: - Each node visitor class visits an op and serializes the data into MPSTensors and MPSNodes which get appended in the flatbuffer - The entries from the FlatBuffer are parsed in the runtime and based on them the MPSGraph is constructed (for more info see `MPSGraphBuilder` class and corresponding ops from `operators/` folder. - This method removes the additional read and writes to disk of the MPSGraphExecutable (once during AOT and once during runtime). **Models summary:** | Model | FP16 | FP32 | | :---: | :---: | :---: | mul | <ul><li>- [x] </li> | <ul><li>- [x] </li> | add_mul | <ul><li>- [x] </li> | <ul><li>- [x] </li> | linear | <ul><li>- [x] </li> | <ul><li>- [x] </li> | edsr | <ul><li>- [x] </li> | <ul><li>- [x] </li> | Mobilebert | <ul><li>- [ ] </li> | <ul><li>- [x] </li> | mv2 | <ul><li>- [x] </li> | <ul><li>- [x] </li> | mv3 | <ul><li>- [x] </li> | <ul><li>- [x] </li> | vit | <ul><li>- [x] </li> | <ul><li>- [x] </li> | w2l | <ul><li>- [x] </li> | <ul><li>- [x] </li> | ic3 | <ul><li>- [x] </li> | <ul><li>- [x] </li> | ic4 | <ul><li>- [x] </li> | <ul><li>- [x] </li> | resnet18 | <ul><li>- [x] </li> | <ul><li>- [x] </li> | resnet50 | <ul><li>- [x] </li> | <ul><li>- [x] </li> | Llama2 | <ul><li>- [ ] </li> | <ul><li>- [x] </li> | emformer_join | <ul><li>- [x] </li> | <ul><li>- [x] </li> | emformer_predict | <ul><li>- [ ] </li> | <ul><li>- [ ] </li> | emformer_transcribe | <ul><li>- [x] </li> | <ul><li>- [x] </li> | dl3 | <ul><li>- [ ] </li> | <ul><li>- [ ] </li> | Pull Request resolved: https://github.com/pytorch/executorch/pull/1655 Reviewed By: cccclai Differential Revision: D52929916 Pulled By: shoumikhin fbshipit-source-id: 8bd2ed124311744ebe19fc17eb0ff508621f974a
D
Denis Vieriu committed
cbc0c95027e580255eba66c6a703041dcefcab31
Parent: 71aedc5
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 1/23/2024, 1:45:34 AM