The oshinko-s2i repository provides Source-to-Image (s2i) images and utilities designed to enable Apache Spark application builders to deploy containerized Spark applications on OpenShift. Written primarily in Shell, the project serves as a foundational component in the radanalytics.io ecosystem for building and managing Spark-based data processing workloads in Kubernetes environments.
The repository distinguishes itself by offering two categories of s2i images: complete images that come with pre-selected Apache Spark distributions baked in during the build process, and incomplete images that include radanalytics.io tooling but allow users to select and install their preferred Spark distribution. This dual approach provides flexibility for different deployment scenarios, enabling teams to either use standardized Spark versions or customize their distributions based on specific requirements.
Building the s2i images is streamlined through Makefiles that support both bulk and individual image construction. Users can build all images at once or target specific language variants including PySpark, Java, and Scala, with both complete and incomplete versions available. The build system allows customization of the image repository through the LOCAL_IMAGE variable, accommodating different registry configurations and organizational requirements.
The project employs the cekit tool for generating image context directories from YAML configuration files, with version 2.2.7 specified as the standard. A provided change-yaml.sh script simplifies the process of modifying component versions for oshinko and Spark within the image definitions. The repository maintains six primary image context directories corresponding to the three language variants in both complete and incomplete forms: pyspark-build, java-build, scala-build, pyspark-build-inc, java-build-inc, and scala-build-inc.
The repository includes several utility scripts that enhance the development and deployment workflow. The make-build-dirs.sh script regenerates context directories from a clean state for pull requests, ensuring consistency and proper artifact management. The get-rad-image.sh script enables R support in the rad-image component, while templates-is.sh modifies templates for use with imagestreams. The release-templates.sh script creates versioned template sets that reference specific oshinko releases, allowing teams to pin stable image versions for production deployments.
Development practices are supported through a git pre-commit hook that prevents commits containing non-zero length tarballs in build directories and warns when YAML or script changes lack corresponding updates to build directories. This automation helps maintain repository integrity and prevents CI test failures related to artifact management.
The project's classification spans multiple domains including containerization, data analytics, Kubernetes, cluster management, stream processing, and elastic computing. Its integration with the broader radanalytics.io ecosystem, combined with support for Apache Spark, Kafka, and real-time analysis capabilities, positions it as a critical tool for organizations deploying big data processing pipelines on OpenShift. The repository maintains connections with related projects through overlapping contributors, indicating active participation in the wider container-based analytics community.
