Repository Issue Activity (beta)

#1748 mt_bench support

#1726 希望在压测的多模态数据集类型中加入支持一次输入多张图片的数据集模式

#1685 swe_bench_verified_agentic 测试hang住

#618 FewCLUE/cluewsc数据集支持

#1742 ticket: Reliability, recovery, and scoped access

#1741 ticket: Artifacts, comparisons, and exports

#1740 ticket: Composite evaluations and suite scoring

#1739 ticket: Benchmark catalog and evaluation suites

#1738 ticket: Performance execution and telemetry metrics

#1737 ticket: Versioned workload profiles and deterministic plans

#1736 ticket: Compass identity models and deployments

#1735 ticket: Compass run lifecycle and persistence

#1734 spec: AI Compute Compass V1

#1706 Add AgentX workload support to EvalScope Perf

#1599 Report list: group same-model reports for display, without merging report directories

#1707 [Bug] longbench_v2 数据集指定 few_shot_num 后报错 KeyError: 'fewshot'

#1709 External agent bridge drops perf_metrics (and reasoning) from the transcript, so collect_perf records nothing for bridge runs

#1708 mv_bench 评测报错，fine_grained_pose 子数据集截断

#951 [🎯 Roadmap] EvalScope Roadmap

#1699 hi，感谢开源工具，请问是否支持自定metric，有些bench不想用acc，例如extract_match等

#1697 Follow-up to #1578: judge prompt placeholder injection ({gold} splice); unseeded bootstrap CI bounds; metric crash recorded as zero with SUCCESS status; allow_inclusion grades by substring

#1693 MCP over HTTP transport always fails with CancelledError; stdio unusable in service mode

#1694 [Bug] evalscope 1.11.1 perf TTFT statistics anomalous and abnormally high (min/P1/P5/P10/P25 all 0, P50 jumps to ~9555ms)

#1695 AgentLoop docstrings point at AgentTrace.final_prediction, which does not exist

#1452 [Feature ] video generate datasets support