mirror of
https://github.com/huggingface/transformers.git
synced 2026-04-01 12:28:47 +00:00
069684ef87
* feat(ci): add continuous batching to benchmarks * refactor(ci): PR comments * refactor(cb): when stopping, block by default * fix(benchmarks): `stream` -> `streaming` * fix(benchmarks): invalid configuration when cb has attn_impl == sdpa * tests(cb): fix attn impl * fix(benchmarks): update `get_throughput` formula * fix(benchmarks): prevent version conflicts and ensure proper cleanup in continuous batching (#42063) * Initial plan * fix(benchmarks): ensure proper cleanup and remove transformers from requirements - Remove transformers from benchmark_v2/requirements.txt to prevent version conflicts - Add try-finally block to ensure ContinuousBatchingManager.stop() is always called - This fixes TypeError about unexpected 'streaming' argument and prevents OOM from improper cleanup Co-authored-by: McPatate <9112841+McPatate@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: McPatate <9112841+McPatate@users.noreply.github.com> * fix(benchmarks): raise the exception on failure instead of ignoring we catch the exception later on and raising it here helps debugging because it will be logged * test(cb): comment out failing tests for now added a `FIXME` mark * fix(benchmarks): revert `finally` removal but keep raising exception * test(cb): fix missing `require_read_token` import * refactor(benchmarks): error if no benchmarks were run * refactor(benchmarks): change default lvls of cb bench config --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: McPatate <9112841+McPatate@users.noreply.github.com>