Robustness fix: Annotation file not dropping likelihood column if present from machine labels (#3323)
* Drop likelihood cols and validate coords Fix format_training_data to handle MultiIndex columns with likelihoods: detect coord level, keep only x/y columns (logging when likelihoods are dropped), and raise if x/y are missing. Use the row values to ensure an even number of coordinates after dropping non-coord columns (error if odd), reshape into (N,2), filter NaNs, and clip out-of-image joints. Also skip images without labels. * Add test to ignore likelihood columns Add a unit test to tests/test_trainingsetmanipulation.py that verifies format_training_data ignores 'likelihood' columns when formatting training data. The test monkeypatches read_image_shape_fast, constructs a DataFrame with inserted likelihood columns after each y coordinate, and compares the formatted outputs (image, size, joints) against a baseline produced from the original x/y-only DataFrame to ensure identical results. * Drop likelihood columns from annotations Add helper _drop_likelihood_columns to remove any 'likelihood' coord columns from MultiIndex annotation DataFrames (using 'coords' level if present or the last level otherwise). Use this sanitizer in merge_annotateddatasets and as a final check in format_training_data to avoid concatenated likelihood columns (fixes issues such as napari-deeplabcut #204 and DeepLabCut #3319). Also log when likelihood columns are detected and dropped, tighten x/y coordinate validation, and apply minor formatting cleanups. * Add test for dropping likelihood columns in merge Add a unit test (tests/test_trainingsetmanipulation.py) that verifies merge_annotateddatasets removes 'likelihood' columns from single-animal annotation data. The test creates a temporary labeled-data HDF with multiindex columns (x/y/likelihood), runs merge_annotateddatasets with a non-multianimal config, and asserts that the returned DataFrame and the saved HDF/CSV outputs no longer contain 'likelihood' while preserving x/y coordinates.
C
Cyril Achard committed
9dbae0aba04e520833dc093f28464a577b409e52
Parent: d84832c
Committed by GitHub <noreply@github.com>
on 5/15/2026, 12:01:51 PM