Repository Issue Activity (beta)

#601 False-positive "distillation" ban on Pro subscriber - automated appeal rejected by bot, no human review available

#593 High Token Inflation and Lack of Representation for Bangladeshi Bengali

#299 Tiktoken educational BPE trainer takes long time to train with vocab size 30k

#464 Add GPT-5.1 with the o200k_base encoding in `tiktoken/model.py`

#578 GPT-5.1 not available

#589 Need wheels for Python 3.15 on Mac

#571 Tik Tok follow updates

#577 pip install tiktoken

#568 [Security] Reachable Rust panic (denial of service) when tokenizing with an untrusted encoder missing single-byte tokens

#179 Please commit your Cargo.lock file in the repository

#366 tiktoken under counting tokens for openai text-embedding-3-large

#521 tiktoken 0.12 cannot handle qwen3.5 in huggingface

#297 GPT4o出现低级bug：发现最新token中的垃圾语料及实测GPT4o胡言乱语出现幻觉

#557 CoreBPE::new has an unconstrained NSE type parameter, making normal downstream calls fail type inference

#169 tiktoken_ext.__pycache__ Issue On Amazone Lambda

#534 build_sdist uploads artifact with empty name (matrix variables undefined)

#15 Extremely Long Text results in PanicException, which is hard to catch in python code

#195 Very slow for inputs like "a" * 100000

#245 Panic (stack overflow) when encoding a certain string

#513 [Bug] tiktoken 0.9.0 ignores TIKTOKEN_OFFLINE and TIKTOKEN_CACHE_DIR, still attempts network download

#550 my_tiktoken_extension ├── tiktoken_ext │   └── my_encodings.py └── setup.py my_tiktoken_extension ├── tiktoken_ext │   └── my_encodings.py └── setup.py

#530 encode_ordinary_batch — reproducible multi-second tail stalls on 32-core box (o200k_base, num_threads=8)

#547 Use Trusted Publishing to upload to PyPI

#474 Token count discrepancy between tiktoken and API response when messages contain tool calls

#250 Enhancement: Add convenience token-counting functions to this package