IBM releases comparative evaluation of its open-source CUGA agent framework
What happened
On 3 November 2026, IBM's team published a comparative evaluation on the Hugging Face blog, testing three agent frameworks — CUGA, Hermes and Deep Agents — on 90 AppWorld tasks using the same open-weight model, GLM-5.3-flash. Task pass rates were 81.1%, 46.7% and 30.0% respectively, and full-scenario pass rates were 66.7%, 26.7% and 6.7%. The comparison set out to examine performance differences when the same model is swapped between harnesses, and CUGA led on both metrics, the report said.
Written by AI from the reports · updated 3 h ago
Timeline
Follow the reports to see the event from each side.
- Hugging Face BlogSelectedSame Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?
An IBM team compared CUGA, Hermes and Deep Agents on 90 AppWorld tasks using the same open-weight model, GLM-5.3-flash. Task pass rates were 81.1%, 46.7% and 30.0%, and full-scenario pass rates were 66.7%, 26.7% and 6.7%.
Heat of this event
Current heat 9·Peak in the comparable range 10（Nov 4 01:00）·Change over 24 hours in the comparable range –
The trend compares only the same accounts observed without a break, so its range can be narrower than the current heat count. Move the pointer or tap the chart to see each hour; with a keyboard, use the left and right arrow keys.