CI failures with dtype-override (#1919)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/1919

language_llama failure: https://www.internalfb.com/sandcastle/workflow/3071454945868675212
llama stories failure: https://www.internalfb.com/intern/testinfra/diagnostics/5066549797810715.562950083098286.1707509873/

Reviewed By: larryliu0820, angelayi

Differential Revision: D53625845

fbshipit-source-id: 59b4af4cd5329c7c8baa199831ebf1d21540cb81