SIGN IN SIGN UP

feat(new codecs): introduce avro (#19342)

* feat(codecs): new `avro` decoder

- cherry-pick from @cameronbraid's PR#17521
- fix some build errors on latest version
- adjust Avro sorting in documentation

* chore(avro decoder): remove unused imports

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): rename member of AvroDeserializerConfig

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): use typedef to enhance readability

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): convert magic number to const value

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): make schema_definition support log_namespace

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): format code

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): merge parse_single and parse

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): capitalize the first word in comments

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): make parse support log_namespace

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* chore(avro decoder): treat uuid as Vec<u8>

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>

* format cue files

* fix handling for timestamp types

* fix error message

* add empty bytes check

* added uuid test

* Revert "chore(avro decoder): treat uuid as Vec<u8>"

This reverts commit 9ff71550c4b9c087a56ccc01e3280aeadca889bc.
Let uuid as String is more common for that avro' uuid type is
a logic type of String.

* chore(avro decoder): remove TODO label

* chore(avro decoder): replace Vrl From<f64> with from_f64_or_zero

* chore(avro codec): add roundtrip test

* chore(avro codec): add readme file for avro test

* chore(avro codec): fix some spell errors in test datas

* chore(avro codec): fix some spell errors in the patch

* chore(avro codec): fix spell errors in avsc

* chore(avro codec): add avsc to spelling/allow.txt

* chore(avro codec): add some ignores for spelling check action

- ignore patch hunk comments
- ignore git index header

* chore(avro codec): add avsc to spelling/allow.txt

* chore(avro codec): rename lib/codecs/tests/data/avro/avro to lib/codecs/tests/data/avro/avro

* chore(avro codec): merge test function roundtrip_avro_fixtures2 and roundtrip_avro_fixtures_reverse2

* chore(avro codec): refactor patch way of test case generating with features

* chore(avro codec): merge avro_generate_test_case into avro.rs

* Revert "chore(avro codec): add some ignores for spelling check action"

This reverts commit a0ac082e736d663c2d63fafefd3fad17db982401.

* chore(avro codec): move fixtures generator to tests/bin

* chore(avro codec): relax avro map test

Due to the sort of map may change, use test case of one item
map to relax the test.

* chore(avro codec): re-enable the map type test case in roundtrip test

* make generate-component-docs

* remove support for decimal

* readd removed test case

* fix errors for decimal

* add unsupported type comment and return err on duration and local timestamps

* change keys to KeyString

* Extend documentation for avro schema

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* refactor test and run fmt

* Update avro readme

Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* remove duration from exclude list as it is no longer supported

* fix casing and update cue files

* fix enum casing

* fmt

* fix flaky test by not having ordering issues in map

* remove decimal var as it is not supported

* remove support for date, fixed, time_millis and timestamps

* change word for spelling

* readd exclude

* readd tests for unsupported types

* ignore decimal test unused

* fmt readme

* add support for timestamp and local timestamp

* remove now supported from unsupported

* update docs

---------

Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
Co-authored-by: Cameron Braid <cameron@drivenow.com.au>
Co-authored-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com>
Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
I
Ion-manden committed
bd2cff83a6df2e0e287c40000bd9d7e9d24a59dc
Parent: b7b8081
Committed by GitHub <noreply@github.com> on 12/15/2023, 9:59:04 AM