chore(buffers): Move benchmarks into 'buffers' sub-crate (#7576)
This commit moves the top-level benchmarks of buffers into the core buffers crate. This is not a simple code move. There are two fundamental changes being proposed here: introduction of quickcheck tests and intentionally spare benchmarks. ## QuickCheck The first major change in this PR are the newly introduce QuickCheck tests for buffers, both for in-memory and on-disk buffers. The test is simplistic: every `T` that is sent into the buffer should come back out in order unless the buffer was over-full, in which was the `WhenFull` condition applies. The model we compare the buffer against is a `VecDeque` with logic hung off it to mimic the [`buffers + num-senders`](https://docs.rs/futures/0.3.15/futures/channel/mpsc/fn.channel.html) quirk of futures' mpsc, our own shedding logic. The test loop is done without reference to any external runtime: we use the bare interface of `Sink` and Stream`. This avoids the non-determinism introduced by a runtime and has the happy side-benefit of making buffers more runtime agnostic. Notes are left in the model test indicating how we could expand the model. ## Intentionally Spare Benchmarks The benchmark code follow a similar tactic to the quickcheck tests: they do not use a runtime and instead use the bare `Sink` and `Stream` API. While criterion does have support for running async/await code their documentation notes that this setup will introduce overhead and noise, undesirable in this core serialization point of vector. The benchmarks are done with respect to the in-memory and on-disk buffer variants, crossed by a write-then-read and write-and-read benchmark variation. Every effort has been taken to drive down noise as much as possible on my development machine and the results are promising, though we'll see how things shake out in the CI system. Perhaps controversially the benchmark code _does not_ concern itself with the correctness of the buffer response, except in the most coarse sense. This is in line with the goal of driving down noise, reducing the amount of code running in the measurement loop, but runs a little counter to the project's previous strategy with regard to benchmarks. The intention here is for the quickcheck tests to tackle correctness issues and for the benchmark code to focus solely on measurement. The benchmark measurement loops are simplistic and fail-fast with regard to error conditions. Resolves #7458 Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
B
Brian L. Troutwine committed
19fb08ab995991cdbc666a287cbee67a144fbbc9
Parent: 7dec4bf
Committed by GitHub <noreply@github.com>
on 5/27/2021, 9:23:07 PM