Repository Issue Activity (beta)

#4038 minerva_math/putnam_axiom/leaderboard-math: thousands-separator comma strip fuses bare digit tuples ("0,1" -> "01")

#4036 minerva_math/putnam_axiom/leaderboard-math: sqrt shorthand normalization corrupts indexed roots \sqrt[3]{x}

#4028 --cache_requests delete fails when the cache directory does not exist

#3046 Cache fails when repeats are greater than 1

#3749 Feature request: canonical reproducibility-bundle export in scripts/

#4031 minerva_math: REMOVED_EXPRESSIONS entry "ft" corrupts \left (substring replace inside control sequences)

#4030 minerva_math: identical answers scored wrong when sympy cannot parse them (tuples, intervals, matrices)

#4002 Scalar `seed` in a YAML config crashes with TypeError; `seed: 0` silently disables seeding

#4011 `_parse_dict_args` is a no-op under postponed annotations, and `DICT_KEYS` is unused

#3962 [Proposal] OpenEval Import/Export Support

#3558 think_end_token support needed for local-chat-completions mode

#3460 max_model_length for gemma3-12b is set to 2048 (_DEFAULT_MAX_LENGTH) with the hf processor

#4022 Feature request: EvalPort import/export for per-document samples and results

#4019 Proposal: Wilson CIs + content-hash integrity field on results

#3017 Support for using a remote /tokenize API endpoint as the tokenizer

#4017 Proposal: Wilson CI bounds, MDE, and optional BH-FDR flags on results output

#4007 Generative-task scorers cannot distinguish an unparseable response from a wrong answer (census: 2,936/4,524 tasks affected)

#4005 --use_cache crashes before inference for multimodal image requests

#3080 Fix Metric Calculation for Repeats

#3493 Standardizing Task Formats

#3881 Request cache key ignores generation_kwargs -> silently reuses cached instances across different sampling parameters

#3498 what's the proper config for aime24 evalated on Qwen3?

#3989 Fix broken Discord link in contributing guide

#3930 [Bug] afrixnli prompt_1: doc_to_text uses .format() braces, so premise and hypothesis are never substituted (35 tasks)

#3859 SCROLLS tasks fail to load with recent datasets (dataset scripts no longer supported)