Repository Issue Activity (beta)

#120 OpenAI-compat: ChatCompletionSampler plus OPENAI_BASE_URL /v1

#117 Optional EvalPort interop for SingleEvalResult / EvalResult

#116 Reproducibility: provenance bundle + paired statistics for fixed-item evals

#115 MATH grading takes the first "Answer:" match (not the final answer), and check_equality interpolates candidate text raw into the grader prompt

#83 Medical category of questions in the dataset

#109 Standard deviations for healthbench are wrong

#103 Question: where would a long-horizon “tension crash test” benchmark best live in your eval ecosystem?

#105 Batching for higher throughput?

#102 Question about healthbench mandarin version

#100 Regarding the environmental data issue of the web search results of BrowseComp?

#99 Answer of a question in browsecomp may be wrong

#98 What dataset did math use?

#97 What to do now that simple-evals won't be updated?

#96 healthbench_eval can not reproduced‌ 0.67 on the gpt-5

#7 Run benchmarks also for GPT-3.5 versions and Claude Sonnet and Haiku

#85 Cost of a Single Evaluation Using GPT-4.1 in HealthBench

#90 Possible Typo in DROP Benchmark Accuracy

#89 GPQA Accuracy Mismatch on Completions vs Responses API (GPT-4o August 2024)

#88 Environment setup file for Healthbench

#28 How do we run this code?

#80 HealthBench asking for azure credentials

#79 Clearly wrong / tampered prompt_id with "milf"

#78 HealthBench performance data json

#77 Incorrect scores in HealthBench?

#76 Access to HealthBench Dataset?