SIGN IN SIGN UP

[executorch][flat_tensor] implement load into and dont hold onto the segment (#8650)

* [flat_tensor] Persist FreeableBuffers of external constants in method

Pull Request resolved: https://github.com/pytorch/executorch/pull/8437

## Problem
Currently, the FlatTensorDataMap persists tensors, and returns a FreeableBuffer with an empty free function.

The NamedDataMap should not persist data, as most cases (eg. delegate) will want it to be freed.

Ownership should be on the caller; `get_data` returns a FreeableBuffer that 'owns' the data. The FreeableBuffer in turn is owned by the caller.

NOTE: this doesn't support the case where we want to share plain tensors between methods/pte files at runtime. A custom NDM could support that use-case.

## This diff:
1. Introduces a 'NamedData' struct to method.h. This holds a key and a FreeeableBuffer.
2. Iterate over all the flatbuffer tensors to count the constants tagged with EXTERNAL.
NOTE: this will increase load time for all users. Potentially allocate chunks of 16 and use a linked list to store external constants, or store this number in PTE file (see D69618283).
3. Allocate space for num_external_constants using the method allocator.
4. Iterate over all flatbuffer tensors and use the named_data_map to resolve EXTERNAL tensors into the array of NamedData.
5. Pass the resolved external constants to tensor_parser, along with NDM (used for mutable external tensors).
6. Resolved external tensors are stored inside method. They are freed when the method is destructed.


Some notes:
https://docs.google.com/document/d/1_PBi4JgODuClUPD4PCUWrKNjyUH54zOUHGUJ3QHDNes/edit?tab=t.0#heading=h.blsvwraxss7g
ghstack-source-id: 267364187

TODO: add test case when two fqns point to the same data buffer.

Differential Revision: [D69477027](https://our.internmc.facebook.com/intern/diff/D69477027/)

* [executorch][flat_tensor] implement load into and dont hold onto the segment

Pull Request resolved: https://github.com/pytorch/executorch/pull/8447

1. Implement load_into in FlatTensorDataMap
2. Do not persist 'data_ro' in the FlatTensorDataMap. From `get_data`, return the FreeableBuffer given by the data loader.

TODO: add test for load_into.
ghstack-source-id: 267467148

Differential Revision: [D69148652](https://our.internmc.facebook.com/intern/diff/D69148652/)

---------

Co-authored-by: lucylq <lfq@meta.com>
H
Hansong committed
728c255fff0e75cfc399b22a7d745627db13e0a1
Parent: 9c51e58
Committed by GitHub <noreply@github.com> on 2/24/2025, 10:43:22 PM