SuperCLUE is a comprehensive benchmark for evaluating foundation models in Chinese across multiple dimensions of capability.
The benchmark addresses the need to systematically assess Chinese language models across a broad spectrum of abilities. SuperCLUE organizes evaluation into four capability quadrants: language understanding and generation, professional skills and knowledge, AI agent functionality, and safety. These quadrants are further refined into twelve foundational abilities. The tool provides multi-dimensional assessment through structured testing that measures how well models perform across these distinct competency areas, with particular emphasis on emerging capabilities like agent-based task planning and tool usage.
Organizations building or selecting Chinese language models should use SuperCLUE when they need comprehensive, standardized evaluation across language understanding, domain knowledge, agent capabilities, and safety dimensions. The benchmark suits projects that require detailed performance comparison across multiple model families and capability areas rather than single-metric assessment. The tool is particularly relevant for teams evaluating models for production deployment where understanding performance across safety, reasoning, and autonomous task execution matters alongside basic language tasks.
The project maintains active development with regular benchmark updates and expanded evaluation frameworks. The tool has grown to include specialized sub-benchmarks addressing specific concerns such as agent evaluation and safety assessment. The project publishes annual comprehensive reports analyzing the state of Chinese AI models and maintains an official website with current leaderboards and detailed evaluation results.
