Hugging Face Blog·· 5 h ago
FairStress: A deeper evaluation of fairness in LLM decision-making
FairStress: A deeper evaluation of fairness in LLM decision-making
AI summary
Today we’re introducing FairStress, a benchmark for measuring demographic fairness in LLMs beyond the choice a model ultimately makes in decisions impacting people. FairStress is available for download, and we’re also building it into Olmo-eval, our open workbench for running and comparing model evaluations.
Selection record
Threshold 60Official, first-handFirst 38Second 42
Not admittedSum of both 80 < twice the threshold 120
- Source tier
- Official, first-hand; this tier's threshold is 60
- Pre-filter
- passed：LLM公平性评测基准与模型行为分析
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: Hugging Face Blog · huggingface.co