HanLP is a multilingual natural language processing library that provides tokenization, part-of-speech tagging, named entity recognition, dependency parsing, and semantic analysis across multiple languages.
HanLP addresses the need for a unified NLP toolkit that handles diverse linguistic tasks across many languages. It builds on PyTorch and TensorFlow 2.x to deliver state-of-the-art deep learning models. The library supports ten joint tasks including tokenization, lemmatization, part-of-speech tagging, dependency parsing, constituency parsing, semantic role labeling, semantic dependency parsing, and abstract meaning representation parsing. It leverages open-access corpora like Universal Dependencies and OntoNotes to enable coverage across numerous languages.
Developers should choose HanLP if they need multilingual NLP capabilities or work with languages beyond English. The tool suits both research and production environments. For end users, HanLP offers two deployment options: lightweight RESTful APIs measured in kilobytes for agile development and mobile applications, and native Python APIs for direct integration. Language-specific models are available for English, Chinese, and Japanese that outperform the multilingual model on those languages. The library requires Python 3.6 or higher, though GPU or TPU acceleration is optional rather than mandatory.
The project maintains active unit testing across its codebase. Documentation is comprehensive, with tutorials covering visualization and annotation guidelines. The tool provides both Python and Java client implementations for the RESTful API interface, allowing developers to choose their preferred integration method.
