SIGN IN SIGN UP

use --use_sdpa_with_kv_cache for 1B/3B bf16 (#5861)

Summary:
Pull Request resolved: https://github.com/pytorch/executorch/pull/5861

We should use this option during exporting 1B/3B models as bf16 because KVCache is always fp32. Otherwise, we see regressed performance for 1B/3B in bf16 format.
ghstack-source-id: 246391007

Reviewed By: mergennachin

Differential Revision: D63871048

fbshipit-source-id: 6b3ff80dbc689a04c70e2fcc5c98698bb74f899b
L
Lunwen He committed
2726bdb87efc956298e683aca4f6ddd0039f6030
Parent: 478a9b6
Committed by Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> on 10/7/2024, 6:12:59 PM