Separate `PhysicalDevice`/`DeviceHandle` from `Adapter.*` (#4187)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/4187

`Adapter.*` has too much going on.
ghstack-source-id: 233090245
exported-using-ghexport

bypass-github-export-checks
bypass-github-pytorch-ci-checks
bypass-github-executorch-ci-checks

Reviewed By: SS-JIA

Differential Revision: D59540131

fbshipit-source-id: 3acdda2ccc56f7b0ceeb48d072cfb38c1a800bf8