feat(new codecs): introduce avro (#19342)
* feat(codecs): new `avro` decoder - cherry-pick from @cameronbraid's PR#17521 - fix some build errors on latest version - adjust Avro sorting in documentation * chore(avro decoder): remove unused imports Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): rename member of AvroDeserializerConfig Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): use typedef to enhance readability Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): convert magic number to const value Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): make schema_definition support log_namespace Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): format code Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): merge parse_single and parse Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): capitalize the first word in comments Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): make parse support log_namespace Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * chore(avro decoder): treat uuid as Vec<u8> Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> * format cue files * fix handling for timestamp types * fix error message * add empty bytes check * added uuid test * Revert "chore(avro decoder): treat uuid as Vec<u8>" This reverts commit 9ff71550c4b9c087a56ccc01e3280aeadca889bc. Let uuid as String is more common for that avro' uuid type is a logic type of String. * chore(avro decoder): remove TODO label * chore(avro decoder): replace Vrl From<f64> with from_f64_or_zero * chore(avro codec): add roundtrip test * chore(avro codec): add readme file for avro test * chore(avro codec): fix some spell errors in test datas * chore(avro codec): fix some spell errors in the patch * chore(avro codec): fix spell errors in avsc * chore(avro codec): add avsc to spelling/allow.txt * chore(avro codec): add some ignores for spelling check action - ignore patch hunk comments - ignore git index header * chore(avro codec): add avsc to spelling/allow.txt * chore(avro codec): rename lib/codecs/tests/data/avro/avro to lib/codecs/tests/data/avro/avro * chore(avro codec): merge test function roundtrip_avro_fixtures2 and roundtrip_avro_fixtures_reverse2 * chore(avro codec): refactor patch way of test case generating with features * chore(avro codec): merge avro_generate_test_case into avro.rs * Revert "chore(avro codec): add some ignores for spelling check action" This reverts commit a0ac082e736d663c2d63fafefd3fad17db982401. * chore(avro codec): move fixtures generator to tests/bin * chore(avro codec): relax avro map test Due to the sort of map may change, use test case of one item map to relax the test. * chore(avro codec): re-enable the map type test case in roundtrip test * make generate-component-docs * remove support for decimal * readd removed test case * fix errors for decimal * add unsupported type comment and return err on duration and local timestamps * change keys to KeyString * Extend documentation for avro schema Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * refactor test and run fmt * Update avro readme Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * remove duration from exclude list as it is no longer supported * fix casing and update cue files * fix enum casing * fmt * fix flaky test by not having ordering issues in map * remove decimal var as it is not supported * remove support for date, fixed, time_millis and timestamps * change word for spelling * readd exclude * readd tests for unsupported types * ignore decimal test unused * fmt readme * add support for timestamp and local timestamp * remove now supported from unsupported * update docs --------- Signed-off-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> Co-authored-by: Cameron Braid <cameron@drivenow.com.au> Co-authored-by: WarmSnowy <17583220+WarmSnowy@users.noreply.github.com> Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>
I
Ion-manden committed
bd2cff83a6df2e0e287c40000bd9d7e9d24a59dc
Parent: b7b8081
Committed by GitHub <noreply@github.com>
on 12/15/2023, 9:59:04 AM