Apache Paimon is a lake format that enables building a realtime lakehouse architecture with Flink and Spark for both streaming and batch operations.
Paimon addresses the challenge of integrating real-time streaming updates with traditional data lake architectures. It combines a lake format with an LSM (log-structured merge) structure, allowing organizations to handle both streaming and batch workloads within a unified system. This approach brings real-time data ingestion capabilities into the lakehouse paradigm, eliminating the need to maintain separate systems for streaming and batch analytics.
The tool suits organizations already invested in Flink or Spark ecosystems that need to support real-time analytics alongside batch processing. It is particularly valuable for teams building data platforms where streaming updates must be reflected immediately in the lake without sacrificing batch query performance. Paimon evolved from Flink Table Store within the Flink community and incorporates design concepts from Apache Iceberg, making it a natural fit for teams familiar with either of those projects.
The project maintains active community engagement through multiple channels including GitHub issues, pull requests, dedicated user and development mailing lists, and a Slack workspace. The codebase is written in Java and requires JDK 8 or 11 for building, with Maven 3.6.3 or later as a dependency.
