Apache Polaris is an open-source catalog specifically designed for Apache Iceberg, a table format for large datasets. Its primary purpose is to provide a central, interoperable metadata store for Iceberg tables, enabling seamless data access and management across various data processing engines. This allows users to work with their Iceberg tables using different tools and platforms without compatibility issues.
The core functionality of Polaris revolves around implementing the Iceberg REST API. This API acts as a standardized interface, allowing different data processing engines to interact with the catalog and access the metadata of Iceberg tables. This interoperability is a key feature, as it allows users to utilize a wide range of platforms, including Apache Doris, Apache Flink, Apache Spark, StarRocks, and Trino, to query and manage their Iceberg data. This eliminates vendor lock-in and provides flexibility in choosing the best tools for specific tasks.
The repository provides the source code for building and running Polaris. It is structured into several modules: `polaris-core` which contains the fundamental entity definitions and core business logic; `polaris-server` which houses the REST API server; and `polaris-eclipselink`, an implementation of the MetaStoreManager interface using EclipseLink. The project is built using Gradle and requires Java 21+ and Docker 27+. The README provides detailed instructions on building, testing, and running Polaris, including commands for building and running tests, assembling the project, and running the server locally. It also includes instructions for running Polaris within Docker and Kubernetes environments, offering flexibility in deployment options.
A significant aspect of Polaris is its support for optional build configurations. Users can enable the EclipseLink extension, which allows for integration with various database backends. Furthermore, the build process allows for specifying additional dependencies for EclipseLink, such as JDBC drivers, providing customization options for different database environments. The documentation also provides guidance on running regression tests to ensure the stability and correctness of the system.
The repository also includes comprehensive documentation, accessible at https://polaris.apache.org. This documentation covers the Polaris management API and the Apache Iceberg REST API, providing users with the necessary information to understand and utilize the catalog effectively. The project also encourages community participation through a dev mailing list and a Zulip chat channel, facilitating discussions and collaboration among developers and users. The project is licensed under the Apache License Version 2.0, ensuring open and free usage.
It's important to note that Apache Polaris is currently undergoing incubation at The Apache Software Foundation (ASF). This means the project is in its early stages of development and is being mentored to ensure it meets the standards of the ASF. While the code is available and usable, its stability and completeness are still evolving. The project's incubation status highlights the commitment to open-source development and community involvement, ensuring a collaborative and transparent approach to building a robust and interoperable catalog for Apache Iceberg.
