Velox is a composable C++ execution engine library created by Meta and developed in partnership with IBM/Ahana, Intel, Voltron Data, Microsoft, ByteDance, and other organizations. It provides reusable, extensible, and high-performance data processing components designed for building data management systems that handle batch, interactive, stream processing, and AI/ML analytical workloads. The library accepts fully optimized query plans as input and executes the described computations, but does not include a SQL parser, dataframe layer, or query optimizer, making it primarily a tool for developers integrating and optimizing compute engines rather than for direct end-user consumption.
The core architecture comprises several high-level components. The Type system provides generic typing supporting scalar, complex, and nested types including structs, maps, and arrays. The Vector module implements an Arrow-compatible columnar memory layout with encodings such as Flat, Dictionary, Constant, and Sequence/RLE, along with lazy materialization and out-of-order write support. Expression Eval delivers a fully vectorized expression evaluation engine for efficient execution on Vector/Arrow encoded data. The Functions component includes vectorized scalar, aggregate, and window function implementations following Presto and Spark semantics. Operators implement relational operations including scans, writes, projections, filtering, grouping, ordering, shuffle/exchange, and various join types. The I/O connector interface supports extensible data sources and sinks with multiple file formats including ORC/DWRF, Parquet, and Nimble, plus storage adapters for S3, HDFS, GCS, ABFS, and local files. Network Serializers enable different wire protocols for network communication, supporting PrestoPage and Spark's UnsafeRow. Resource Management provides primitives for handling computational resources including memory arenas, buffer management, tasks, drivers, thread pools, spilling, and caching.
Extensibility is a defining characteristic, allowing developers to define custom types, simple and vectorized functions, aggregate functions, window functions, operators, file formats, storage adapters, and network serializers. The repository includes examples demonstrating extensibility and integration with various component APIs, along with comprehensive developer guides and documentation available at velox-lib.io.
GitGenius activity data reveals the project maintains exceptionally responsive issue and pull request handling, with a median response latency of 0.0 hours and a mean of 2.3 hours across 1462 tracked items. The most active issue label is rebase-request with 1451 occurrences, followed by enhancement and iceberg labels. Primary contributors tracked include prestodb-ci with 4627 events, zhouyuan with 51 events, and yingsu00 with 29 events. The repository shares overlapping contributors with prestodb/presto and duckdb/duckdb, indicating active cross-project collaboration within the data processing ecosystem.
The project supports multiple platforms and compilers with minimum requirements of GCC 11 or Clang 15 on Linux and Clang 15 on macOS. Recommended configurations include CentOS 9/RHEL 9 with GCC 12 and Ubuntu 22.04 with GCC 11. The build system requires CPU support for instruction sets including bmi, bmi2, and f16c, with additional optimization for avx, avx2, and sse on Intel or Neon on ARM architectures. Velox is licensed under Apache 2.0 and maintains an active community through the Velox-OSS Slack workspace, GitHub issues, and discussions, with technical governance and maintainer information documented on the project website.
