Qualcomm AI Engine Direct - Enable zero copy feature (#2531)
Summary: - Add argument "shared_buffer" into compiler_spec, qnn_executor_runner and test scripts - Actually, shared_buffer should be a runtime option since user are responsible to allocate memory for tensors on device. But it seems to have no way to set the runtime option to QnnBackend. Therefore, we put it to compile_spec for now. - Implement SharedBuffer to allocate and free RPC memory - Add QnnMemManger to register shared buffer for tensor - During exection time, we will register memory of tensor data for QNN. And we will deregister them during destruction time of QnnBackend - Add two API `void* QnnExecuTorchAllocCustomMem(size_t bytes, size_t alignment)` and `void QnnExecuTorchFreeCustomMem(void* buffer_ptr)` to allocate RPC memory with SharedBuffer - Users are responsible to allocate "enough" tensor bytes, and set alignment as MemoryAllocator::kDefaultAlignment. See runtime/core/memory_allocator.h. Pull Request resolved: https://github.com/pytorch/executorch/pull/2531 Reviewed By: kirklandsign Differential Revision: D55142607 Pulled By: cccclai fbshipit-source-id: 6a394ef0023e70362c3cf963a0519c3efaa95bc2
S
shewu-quic committed
a531ca5a5ab5b5e503739a40a7cbec2e46767f0b
Parent: 9e922d3
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
on 3/25/2024, 8:39:44 AM