Repository Issue Activity (beta)

#299 GPQA debug mode ignores an explicit --examples sample count

#298 Eval CLI --examples crashes non-debug AIME and GPQA because n_repeats stays 8

#296 Browser HTML conversion can leak global html2text monkey patches after an exception

#294 AIME and GPQA treat num_examples=0 as the full dataset

#292 HealthBench debug mode ignores an explicit --examples override

#290 Distributed chat executes each tool call independently on every model-parallel rank

#288 Eval progress helper crashes on an empty sample set

#286 Local Harmony tokenizer assigns token ID 200018 to two special tokens

#283 Nullable Responses request fields can crash the local server

#281 Responses stub backend does not reset fake output on new requests

#279 Responses Ollama backend injects token 0 when no streamed token is ready

#277 Responses API default checkpoint path is not expanded

#275 Responses vLLM backend passes TP environment value as a string

#273 Responses API Triton backend binds CUDA devices using global rank

#271 Responses API Triton backend creates KV caches on CPU

#270 Distributed Torch and Triton sampling can diverge across model-parallel ranks

#268 Text generation default unlimited limit passes an invalid max_tokens value

#266 Distributed Torch inference binds CUDA devices using global rank

#264 Chat Completions reasoning is injected into eval prompt history

#262 Eval samplers retry permanent API errors forever

#260 HealthBench bootstrap uncertainty is not reproducible

#257 Merged eval summary mislabels HealthBench subset names

#256 HealthBench usage metadata crashes with the chat_completions sampler

#254 ABCD eval grader can score an earlier answer instead of the model's final answer

#240 MCP mutation probe - please ignore