IBM releases comparative evaluation of its open-source CUGA agent framework

Developing1 reports1 sourcesupdated 2 h ago

What happened

AI digest

On 3 November 2026, IBM's team published a comparative evaluation on the Hugging Face blog, testing three agent frameworks — CUGA, Hermes and Deep Agents — on 90 AppWorld tasks using the same open-weight model, GLM-5.3-flash. Task pass rates were 81.1%, 46.7% and 30.0% respectively, and full-scenario pass rates were 66.7%, 26.7% and 6.7%. The comparison set out to examine performance differences when the same model is swapped between harnesses, and CUGA led on both metrics, the report said.

Written by AI from the reports · updated 1 h ago

LatestNov 3 23:57

IBM publishes evaluation: with the same model, CUGA reaches an 81.1% pass rate on AppWorld tasks, ahead of Hermes and Deep Agents.

Timeline

Follow the reports to see the event from each side.

Nov 3

23:57
Hugging Face BlogSelected
Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?
An IBM team compared CUGA, Hermes and Deep Agents on 90 AppWorld tasks using the same open-weight model, GLM-5.3-flash. Task pass rates were 81.1%, 46.7% and 30.0%, and full-scenario pass rates were 66.7%, 26.7% and 6.7%.

Heat of this event

Not enough continuous observations yet to draw a trend.