Hugging Face Blog·· 5 d ago
Token Pooling-Aware Training for Multi-Vector Embedding Models
Token Pooling-Aware Training for Multi-Vector Embedding Models
AI summary
Multi-vector embedding models give each query token its own opportunity to find a match in a document. With pool_factor=4 , my normally trained model retains 94.6% of its unpooled NanoBEIR nDCG@10, with a 4x reduction in document vectors. Token pooling-aware training raises that retention to 97.8% at the same storage budget!
Selection record
Threshold 60Official, first-handFirst 34Second 38
Not admittedSum of both 72 < twice the threshold 120
- Source tier
- Official, first-hand; this tier's threshold is 60
- Pre-filter
- passed：多向量嵌入模型Token Pooling训练技术
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: Hugging Face Blog · huggingface.co