BenCao is an instruction-tuned large language model collection specialized for Chinese medical knowledge and question-answering.
The project addresses the need for medical domain expertise in large language models by constructing a Chinese medical instruction-tuning dataset built from medical knowledge graphs, medical literature, and ChatGPT API outputs. It then applies instruction-tuning to various base models including LLaMA, Alpaca-Chinese, Bloom, and Huozi to improve their performance on medical question-answering tasks. The approach uses LoRA fine-tuning in half-precision to balance computational efficiency with model performance.
Developers working on Chinese medical AI applications should consider this tool if they need models specifically optimized for healthcare domain tasks in Chinese. The project provides multiple base model options, allowing teams to choose based on their computational constraints and performance requirements. Those with limited resources can leverage the LoRA-based approach, which reduces training overhead compared to full fine-tuning while maintaining effectiveness.
The project demonstrates sustained research engagement with peer-reviewed publications accepted to major venues and continued model releases incorporating new base models as they become available. Development activity includes regular updates introducing improved base models and refinements to the instruction-tuning methodology. The team has published technical documentation on tuning methods for large language models in healthcare contexts and research papers on knowledge-enhanced fine-tuning approaches for reliable Chinese medical response generation.
