Arroyo is a distributed stream processing engine written in Rust that executes stateful computations on bounded and unbounded data streams with subsecond latency.
The tool addresses the need to perform complex analytical queries on high-volume real-time data. It uses SQL as a first-class interface, allowing users to define streaming pipelines declaratively rather than through imperative code. The engine implements the Dataflow model for time-oriented stream processing, supporting stateful operations like windowing and joins, and provides state checkpointing to enable fault tolerance and recovery.
Arroyo suits teams building real-time analytics, fraud detection, security monitoring, or ML feature generation pipelines who want to avoid the operational complexity of traditional streaming systems. It targets users who may not be streaming experts, emphasizing clean separation between pipeline APIs and internal implementation details. The README positions it against Apache Flink, Spark Streaming, and Kafka Streams, highlighting three distinguishing points: serverless-first design for cloud environments with automatic scaling and rescheduling, SQL as a high-performance first-class concern rather than an add-on, and a focus on accessibility for non-experts over low-level control.
The project maintains active development with regular commits and passing continuous integration. Pull requests are welcomed according to the repository guidelines.
