feat: adds Component Inputs telemetry (#10254)

* feat: Introduce telemetry tracking for sensitive field types

Added a new set of field types that should not be tracked in telemetry due to their sensitive nature, including PASSWORD, AUTH, FILE, CONNECTION, and MCP. Updated relevant input classes to ensure telemetry tracking is disabled for these sensitive fields, enhancing data privacy and security.

* feat: Enhance telemetry payloads with additional fields and serialization support

Added new fields to the ComponentPayload and ComponentInputsPayload classes, including component_id and component_run_id, to improve telemetry data tracking. Introduced a serialize_input_values function to handle JSON serialization of component input values, ensuring robust handling of input data for telemetry purposes.

* feat: Implement telemetry input tracking and caching

Added functionality to track and cache telemetry input values within the Component class. Introduced a method to determine if inputs should be tracked based on sensitivity and an accessor for retrieving cached telemetry data, enhancing the robustness of telemetry handling.

* feat: Add logging for component input telemetry

Introduced a new method, log_package_component_inputs, to the TelemetryService for logging telemetry data related to component inputs. This enhancement improves the tracking capabilities of the telemetry system, allowing for more detailed insights into component interactions.

* feat: Enhance telemetry logging for component execution

Added functionality to log component input telemetry both during successful execution and error cases. Introduced a unique component_run_id for each execution to improve tracking. This update ensures comprehensive telemetry data collection, enhancing the robustness of the telemetry system.

* feat: Extend telemetry payload tests and enhance serialization

Added tests for the new component_id and component_run_id fields in ComponentPayload and ComponentInputsPayload classes. Introduced a new test suite for ComponentInputTelemetry, covering serialization of various data types and handling of edge cases. This update improves the robustness and coverage of telemetry data handling in the system.

* fix: Update default telemetry tracking behavior in BaseInputMixin

Changed the default value of track_in_telemetry from True to False in the BaseInputMixin class. Updated documentation to clarify that telemetry tracking is now opt-in and can be explicitly enabled for individual input types, enhancing data privacy and control.

* fix: Update telemetry tracking defaults for input types

Modified the default value of `track_in_telemetry` for various input classes to enhance data privacy. Regular inputs now default to False, while safe inputs like `IntInput` and `BoolInput` default to True, ensuring explicit opt-in for telemetry tracking. Updated related tests to reflect these changes.

* feat: add chunk_index and total_chunks fields to ComponentInputsPayload

This commit adds two new optional fields to ComponentInputsPayload:
- chunk_index: Index of this chunk in a split payload sequence
- total_chunks: Total number of chunks in the split sequence

Both fields default to None and use camelCase aliases for serialization.
This is Task 1 of the telemetry query parameter splitting implementation.

Tests included:
- Verify fields exist and can be set
- Verify camelCase serialization aliases work correctly
- Verify fields default to None when not provided

Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* refactor: update ComponentInputsPayload to support automatic splitting of oversized inputs

This commit enhances the ComponentInputsPayload class by implementing functionality to automatically split input values into multiple chunks if they exceed the maximum URL size limit. Key changes include:

- Added methods for calculating URL size, truncating oversized values, and splitting payloads.
- Updated component_inputs field to accept a dictionary instead of a string for better handling of input values.
- Improved documentation for the ComponentInputsPayload class to reflect the new splitting behavior and usage examples.

These changes aim to improve telemetry data handling and ensure compliance with URL length restrictions.

* refactor: enhance log_package_component_inputs to handle oversized payloads

This commit updates the log_package_component_inputs method in the TelemetryService class to split component input payloads into multiple requests if they exceed the maximum URL size limit. Key changes include:

- Added logic to split the payload using the new split_if_needed method.
- Each chunk is queued separately for telemetry logging.

These improvements ensure better handling of telemetry data while adhering to URL length restrictions.

* refactor: centralize maximum telemetry URL size constant

This commit introduces a centralized constant, MAX_TELEMETRY_URL_SIZE, to define the maximum URL length for telemetry GET requests. Key changes include:

- Added MAX_TELEMETRY_URL_SIZE constant to schema.py for better maintainability.
- Updated split_if_needed method in ComponentInputsPayload to use the new constant instead of a hardcoded value.
- Adjusted the TelemetryService to reference the centralized constant for URL size limits.

These changes enhance code clarity and ensure consistent handling of URL size limits across the telemetry service.

* refactor: update ComponentInputsPayload tests to use dictionary inputs

This commit modifies the tests for ComponentInputsPayload to utilize a dictionary for component inputs instead of a serialized JSON string. Key changes include:

- Renamed the test method to reflect the new input type.
- Removed unnecessary serialization steps and assertions related to JSON strings.
- Added assertions to verify the correct handling of dictionary inputs.

These changes streamline the testing process and improve clarity in how component inputs are represented.

* test: add integration tests for telemetry service payload splitting

This commit introduces integration tests for the TelemetryService to verify its handling of large and small payloads. Key changes include:

- Added tests to ensure large payloads are split into multiple chunks and queued correctly.
- Implemented a test to confirm that small payloads are not split and result in a single queued event.
- Created a mock settings service for testing purposes.

These tests enhance the reliability of the telemetry service by ensuring proper payload management.

* test: enhance ComponentInputsPayload tests with additional scenarios

This commit expands the test suite for ComponentInputsPayload by adding various scenarios to ensure robust handling of input payloads. Key changes include:

- Introduced tests for calculating URL size, ensuring it returns a positive integer and accounts for encoding.
- Added tests to verify the splitting logic for large payloads, including checks for chunk metadata and preservation of fixed fields.
- Implemented property-based tests using Hypothesis to validate that all chunks respect the maximum URL size and preserve original data.

These enhancements improve the reliability and coverage of the ComponentInputsPayload tests, ensuring proper functionality under various conditions.

* [autofix.ci] apply automated fixes

* optimize query param encoding

Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>

* refactor: extract telemetry logging logic into a separate function

This commit introduces a new function, _log_component_input_telemetry, to centralize the logic for logging component input telemetry. The function is called in two places within the generate_flow_events function, improving code readability and maintainability by reducing duplication. This change enhances the clarity of telemetry handling in the flow generation process.

* refactor: optimize truncation logic in ComponentInputsPayload

This commit refines the truncation logic for input values in the ComponentInputsPayload class. The previous binary search method for string values has been simplified, allowing for direct truncation of both string and non-string values. This change enhances code clarity and maintains functionality while ensuring optimal handling of oversized inputs.

* refactor: update telemetry tracking logic to respect opt-in flag

This commit modifies the telemetry tracking logic in the Component class to change the default behavior of the `track_in_telemetry` attribute from True to False. This adjustment enhances user privacy by requiring explicit consent for tracking input objects in telemetry. The change ensures that sensitive field types are still auto-excluded from tracking, maintaining the integrity of the telemetry data.

* refactor: update tests to use dictionary format for component inputs

This commit modifies the integration tests for telemetry payload validation and component input telemetry to utilize dictionaries for component inputs instead of serialized JSON strings. Key changes include:

- Updated assertions to compare dictionary inputs directly.
- Enhanced clarity and maintainability of the test cases by removing unnecessary serialization steps.

These changes improve the representation of component inputs in tests, aligning with recent refactoring efforts.

* [autofix.ci] apply automated fixes

* refactor: specify type for current_chunk_inputs in ComponentInputsPayload

This commit updates the type annotation for the current_chunk_inputs variable in the ComponentInputsPayload class to explicitly define it as a dictionary. This change enhances code clarity and maintainability by providing better type information for developers working with the code.

* test: add component_id to ComponentPayload tests

This commit enhances the test cases for the ComponentPayload class by adding a component_id parameter to various initialization tests. The updates ensure that the component_id is properly tested across different scenarios, including valid parameters, error messages, and edge cases. This change improves the robustness of the tests and aligns with recent updates to the ComponentPayload structure.

* [autofix.ci] apply automated fixes

* feat: add component_id to ComponentPayload in build_vertex function

* fix: update MAX_TELEMETRY_URL_SIZE to 2048 and adjust related tests

This commit increases the maximum URL size for telemetry GET requests from 2000 to 2048 bytes to align with Scarf's specifications. Corresponding test assertions have been updated to reference the new constant, ensuring consistency across the codebase.

* [autofix.ci] apply automated fixes

* feat(telemetry): add track_in_telemetry field to starter project configurations

* refactor(telemetry): remove unused blank line in test imports

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* [autofix.ci] apply automated fixes (attempt 3/3)

* update starter templates

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* [autofix.ci] apply automated fixes (attempt 3/3)

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>