Apache Pinot is a real-time distributed OLAP datastore engineered to deliver scalable analytics with low latency on massive datasets. Written in Java, it was originally built by engineers at LinkedIn and Uber to power interactive real-time analytic applications. The system is designed to scale horizontally with no upper bound, maintaining constant performance based on cluster size and expected query throughput.
The platform supports dual ingestion modes, accepting batch data from sources like Hadoop HDFS, Amazon S3, Azure ADLS, and Google Cloud Storage, as well as streaming data from Apache Kafka, Apache Pulsar, and AWS Kinesis. This hybrid capability allows organizations to combine batch and streaming sources into unified tables for querying. Pinot also supports upsert operations during real-time ingestion, enabling at-scale data updates with consistency guarantees.
Pinot's query capabilities center on a standard SQL interface accessible through a built-in query editor and REST API. The system can filter and aggregate petabyte-scale datasets with P90 latencies in the tens of milliseconds, making it suitable for interactive UI applications. It supports versatile joins including arbitrary fact-to-dimension and fact-to-fact operations on petabyte datasets. The architecture is column-oriented with various compression schemes such as Run Length and Fixed Bit Length encoding.
The indexing system is pluggable, offering multiple technologies including timestamp, inverted, StarTree, Bloom filter, range, text search, JSON, and geospatial indexes. Built-in multitenancy enables data management and security across isolated logical namespaces, supporting cloud-friendly resource allocation. The platform is cloud-native on Kubernetes with Helm charts providing horizontally scalable and fault-tolerant clustered deployments.
Pinot is particularly well-suited for executing real-time OLAP queries on immutable data with fast aggregations and analytics. It excels at querying time-series data with numerous dimensions and metrics. The system is designed for contexts requiring both real-time stream ingestion and batch processing while maintaining consistent low-latency query performance. Building Pinot uses Maven, with optimized development builds available through the pinot-fastdev profile for faster iteration cycles.
