text2vec is a Python library that converts text into vector representations using multiple embedding and similarity computation models.
The tool addresses the need to transform text at various granularities—words, sentences, and paragraphs—into numerical vectors suitable for machine learning tasks. It implements several established approaches including Word2Vec, RankBM25, BERT, Sentence-BERT, and CoSENT, allowing developers to compute text similarity and perform semantic matching without building these models from scratch. The library is designed for immediate use with pre-trained models available through integration with model hosting platforms.
Developers should choose this tool when building applications requiring text similarity calculations, semantic search, or sentence embeddings in Python environments. It suits projects that need multiple embedding strategies available in a single package, particularly those working with Chinese text or multilingual content where specialized models have been trained. The tool includes command-line utilities for batch processing text vectorization and supports distributed inference across multiple GPUs and CPUs, making it practical for production workloads beyond simple prototyping.
The project maintains active development with regular releases introducing new capabilities. Recent work has focused on expanding model support, including training infrastructure for multiple GPUs and new pre-trained models optimized for Chinese and multilingual semantic matching tasks. The maintainers have published custom-trained models addressing specific language pairs and matching scenarios, indicating ongoing refinement based on evaluation results across test datasets.
