EN
Join waitlist
●  DimAgent Bench · v1.0

Models,
measured on
real work.

Every model below runs inside the DimAgent harness. Historical audited runs and newly collected hand-run scores are labeled by source. Conditions and limits are stated; unrecorded fields stay blank.

Updated 995 task slots 7 benchmark suites models
Top of the boardOne leader per suite
01 / Leaderboard

Who gets the job done.

Historical suites use the best completed run per model; FrontierHarness uses model × effort, 3-run averages. SWE-bench Verified and DeepSWE show current hand-run scores per model × effort. Metrics are not comparable across suites.
Sources: audited history through 2026-10-12 · hand-run scores updated 2026-10-13
02 / Efficiency

More done, for less.

Only FrontierHarness currently records cost and time. Results below are three-run averages under labeled conditions, not a team-vs-single-agent comparison.

Pass rate vs. cost per pass

Up and to the left is better · recorded USD.

Three runs, same config.

3-run averageBest of these 3 runs
Cost per pass · high effort
$1.44 → $0.19
↘ 7.7×
Pass rate · 3-run avg
40.0% → 51.1%
↗ +11.1 pts
03 / Methodology

How we score.

Open tasks, fixed harness, reproducible runs. No leaderboard-specific prompting — models use the same configuration within a suite; runtime and effort may differ across historical runs.

Real work tasks

Seven suites covering terminal, document, code and frontier work — 995 task slots in total (not necessarily distinct tasks).

  • Terminal-Bench 2.1 — 89 tasks × 5 attempts
  • OfficeQA Pro — 133 questions · WorkBuddy — 80 code + 50 office jobs × 3
  • FrontierHarness — frozen 30 tasks (21 TB + 9 DeepSWE); SWE-bench Verified — 500 tasks; DeepSWE v1.1 — 113 tasks (both k=1).

Same team, swapped brain

Each suite runs on the DimAgent harness with the model under test swapped in.

  • Tools and task sandbox shared within each suite
  • Effort, runtime and dates shown where historical records are available
  • No single-agent baseline was measured

Verified outcomes

Task verifiers grade the historical suites; OfficeQA is tolerance-graded. SWE-bench Verified and DeepSWE are hand-run score summaries without raw run artifacts, so only recorded conditions are reported.

  • Incomplete, recovery and merged runs excluded
  • Recorded USD, not invoices (FrontierHarness only)
  • Sources: audited history through 2026-10-12; hand-run summary updated 2026-10-13.

Put the top model
on your team.

DimAgent routes each step to the right model automatically.

Download full report Join waitlist