Apache Kafka is an open-source distributed event streaming platform maintained by the Apache Software Foundation and used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. The repository is written primarily in Java with Scala components and serves as the central codebase for a platform designed to handle real-time data at scale across distributed systems.
The project is built and tested against Java versions 17 and 25, with the release parameter in javac set to 11 for clients and streams modules to ensure compatibility with their minimum Java versions, while other modules use release parameter 17. Scala 2.13 is the only supported Scala version in Apache Kafka. The build system uses Gradle and supports multiple development workflows, including building JAR files, source JARs, aggregated javadoc, and running comprehensive unit and integration tests. Developers can run particular tests multiple times, execute specific test methods, and adjust logging levels through log4j2.yaml configuration files located in module test resource directories.
The repository includes extensive testing infrastructure with support for test retries through Gradle configuration, allowing failed tests to be retried with configurable limits. Test coverage reports can be generated for the entire project or individual modules, with coverage reports for the core module located in core/build/reports/scoverageTest/index.html and other modules in their respective jacoco test directories. The build system supports generating binary release gzipped tarballs found in core/build/distributions/, and includes functionality for rebuilding RPC auto-generated message data when switching between branches.
Code quality is maintained through multiple analysis tools integrated into the build process. Checkstyle enforces consistent coding style across the codebase with warnings reported in HTML format in subproject build directories, and the build fails if Checkstyle violations are detected. Spotless handles import order optimization as part of static checks, while SpotBugs performs static analysis to identify potential bugs in the code, generating both HTML and XML reports. The project uses JMH microbenchmarks to produce reliable performance measurements within the JVM environment.
The repository supports multiple development environments including IntelliJ IDEA, which natively supports Gradle and automatically checks Java syntax and compatibility for each module, and Eclipse, which requires running the eclipse Gradle task with a configured build directory at project_dir/build_eclipse to avoid conflicts with Kafka's scripts directory. The Streams quickstart archetype artifact can be published to Maven using the mvn deploy command with appropriate Maven settings configuration. Developers can install all projects or specific projects to the local Maven repository, build test JARs, and analyze dependencies using Gradle's dependency debugging tasks including allDeps and allDepInsight for recursive iteration through subprojects. The platform is classified across multiple domains including publish-subscribe messaging, fault-tolerant systems, real-time data streaming, event processing, message queuing, low-latency messaging, distributed systems, stream processing, event-driven architecture, and data integration, reflecting its comprehensive role in modern data infrastructure.