feat(vector sink): Add zstd compression support (#24917)

* chore(deps): enable tonic zstd feature

Enable the zstd compression feature in the tonic dependency to support
zstd compression for gRPC communication in the Vector sink and source.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(vector sink): add zstd compression support

Add support for zstd compression to the Vector sink, alongside the
existing gzip compression. Changes include:

- Update compression field from bool to Compression enum with backward
  compatibility via bool_or_compression deserializer
- Support compression algorithms: "none", "gzip", "zstd"
- Add validation that returns clear errors for unsupported compression
  types (zlib, snappy)
- Apply appropriate tonic::codec::CompressionEncoding based on
  selected algorithm

The default behavior (no compression) is unchanged, ensuring full
backward compatibility with existing configurations.

Resolves #23030

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat(vector source): accept zstd compressed requests

Update the Vector source to accept zstd-compressed data in addition to
gzip. This enables the source to receive and decompress data from Vector
sinks using either compression algorithm, ensuring seamless
interoperability.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* docs: add changelog entry for vector sink zstd compression

Add changelog fragment documenting the new zstd compression support
in the Vector sink, including usage examples for all configuration
formats (legacy boolean, string, and advanced object syntax).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix(vector source): support zstd decompression in gRPC layer

The custom DecompressionAndMetrics tower layer only supported gzip,
rejecting zstd requests before tonic could handle them. Add zstd
support to the decompression layer and a test for the full path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add deprecation entry for vector sink boolean compression syntax

The boolean compression syntax (`compression: true/false`) in the vector
sink is deprecated in favour of the string syntax (`compression: "gzip"`,
`compression: "none"`, etc.), to be removed in v0.57.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(sources): centralize gRPC compression negotiation in shared layer

The shared DecompressionAndMetricsLayer now owns gRPC compression
negotiation end-to-end: after decompressing the request body it strips
the `grpc-encoding` header so tonic treats the request as uncompressed,
and it injects `grpc-accept-encoding: gzip,zstd,identity` into every
response. Per-service `.accept_compressed(..)` calls are removed from
the `vector` and `opentelemetry` sources.

As a consequence, the `opentelemetry` source now transparently accepts
`zstd`-compressed OTLP requests in addition to `gzip`, matching the
capability advertised via `grpc-accept-encoding` and the capability
surface of the `vector` source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf(vector source): avoid intermediate buffer in zstd gRPC decompression

Replace decode_all + extend_from_slice with copy_decode, which streams
directly into output_buf. Eliminates the temporary Vec<u8> that
decode_all allocates, reducing peak memory from compressed + 2×decompressed
to compressed + decompressed per message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(vector sink): parameterize grpc compression tests

Refactor run_sink_test into run_sink_test_with_compression, asserting the
grpc-encoding header and decompressing the received frame when a scheme
is set. Adds gzip, zstd, and none variants of deliver_message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test(vector source): parameterize grpc compression tests

Refactor run_test into a helper parameterized on Option<&str> compression, mirroring the sink-side change and covering no-compression, gzip, and zstd variants.

* refactor(vector source): tie grpc-accept-encoding to encoding enum

Replace the hand-maintained GRPC_ACCEPT_ENCODING_VALUE literal and its
parallel match arms with an AdvertisedEncoding enum whose ALL constant
drives both the advertised header value (built once via LazyLock) and
the parse/to_scheme mapping. Adding a new advertised scheme now
requires adding an enum variant, which makes the to_scheme match
non-exhaustive until it's wired up — the type system enforces what was
previously a convention.

* fix(vector source): reject compressed gRPC frames with no negotiated encoding

When grpc-encoding is identity or absent, from_encoding_header returns None.
The decompression loop previously fell back to gzip via unwrap_or, so a frame
with the compressed bit set would be silently decoded as gzip instead of
rejected. Per the gRPC compression spec the compressed flag requires a
negotiated encoding, so return an error when it is set without one and
convert the now-unreachable fallback into an expect.

* docs(changelog): add authors line to 24917 otel zstd fragment

The fragment was missing the authors: line required by
scripts/check_changelog_fragments.sh.

* refactor(vector source): bind decompressor via match to drop expect

Replace the is_none/assign/as_mut().expect() pattern with a match on &mut decompressor that
uses Option::insert on the None arm. Both arms yield &mut Decompressor directly, so the post-hoc
expect("decompressor must be set") is no longer needed.

* Fix clippy

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
Co-authored-by: Thomas <thomas.schneider@datadoghq.com>