xan is a command-line tool for processing CSV files directly from the shell.
The tool addresses the need to manipulate large CSV files efficiently without loading them entirely into memory. It achieves this through a SIMD-based CSV parser written in Rust and parallelizes computations via multithreading to maximize performance. The tool exposes a collection of composable commands for previewing, filtering, slicing, aggregating, and sorting CSV data. It includes its own expression language tailored for CSV operations that executes faster than dynamically-typed languages, enabling complex transformations beyond what simple commands can accomplish.
Teams working with large datasets in social sciences, web data collection, and analysis should consider xan. It handles gigabyte-scale files efficiently and extends beyond standard CSV manipulation to include utilities for lexicometry, graph theory, and scraping. The tool processes CSV-adjacent formats from specialized domains such as web archival and bioinformatics, and can convert to and from formats including JSON, Excel, and NumPy arrays. It also provides terminal-based visualization capabilities including histograms, scatterplots, heatmaps, and time series displays for data exploration.
The project maintains active development with regular commits across its codebase. Pull requests receive timely review and merging, indicating responsive maintainers. Issues are addressed consistently, with bug reports and feature requests processed regularly. The tool continues to receive enhancements and refinements to its command set and capabilities.
