Overview
Apache StreamPipes is a self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze, and explore IoT data streams. My contributions to this project focus on extending its processing capabilities and enhancing data sink reliability.
Key Contributions
Switch Operator Processors
Merged into Apache StreamPipes [PR #3640]
Introduced a set of “Switch” operator processors to simplify pipeline logic. Previously, users had to chain multiple boolean conditions or write custom JavaScript to implement complex branching logic.
- Boolean Input Switch: Routes events based on true/false values.
- String Input Switch: Routes events based on string matches (e.g., “case A”, “case B”).
- Numerical Input Switch: Routes events based on numerical thresholds or values.
These processors significantly reduce the complexity of pipelines requiring conditional routing.
Enhanced REST Data Sink
Merged into Apache StreamPipes [PR #3541]
Revamped the generic REST Data Sink to support production-grade requirements and migrated it to the new IDataSinkInterface.
- Custom Headers: Added support for configuring custom HTTP headers, enabling use cases like authenticated webhooks.
- Reliability: Implemented retry logic with exponential backoff for failed requests.
- Connection Management: Added timeout configurations and improved connection pooling.
- Observability: Enhanced logging for easier debugging of failed delivery attempts.
These improvements make the REST sink suitable for critical integration scenarios where data delivery guarantees are essential.