Hugging Face Blog·· 3 h ago
Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?
Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?
AI summary
An IBM team compared CUGA, Hermes and Deep Agents on 90 AppWorld tasks using the same open-weight model, GLM-5.3-flash. Task pass rates were 81.1%, 46.7% and 30.0%, and full-scenario pass rates were 66.7%, 26.7% and 6.7%.
Selection record
Threshold 60Official, first-handFirst 62Second 58
AdmittedSum of both 120 ≥ twice the threshold 120
- Source tier
- Official, first-hand; this tier's threshold is 60
- Pre-filter
- passed：对比AI Agent框架并评测任务完成率
- Why it was chosen
- Changing only the harness moved task completion from 30% to 81%, showing how much harness choice shapes an agent's real-world performance.
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: Hugging Face Blog · huggingface.co