Simon Willison·· 2026-10-09
ttok 1.0
ttok 1.0
AI summary
Release: ttok 1.0 I released ttok 0.4 , ran uv tool upgrade ttok , piped a file into the new version... and realized that it was defaulting to the GPT-4 tokenizer when it should very clearly default to GPT-5/GPT-6 instead! I figured switching the default was a reasonable excuse to finally ship a 1.0.
Selection record
Threshold 76Media and individualsFirst 22Second 18
Not admittedSum of both 40 < twice the threshold 152
- Source tier
- Media and individuals; this tier's threshold is 76
- Pre-filter
- passed：发布AI分词器工具ttok 1.0，涉及GPT模型tokenizer
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: Simon Willison · simonwillison.net