All pull requests

Pull requests

• Enforce ProgramBench offline workspaces

  #781·kexinoh opened Sep 18, 2026·

  ·

• fix(eval): wait for LiteLLM spend to settle before recording proxy_cost

  #780·simonrosenberg opened Sep 4, 2026Member·

  ·
  1

• build(deps): bump the version-all group across 1 directory with 43 updates

  dependenciespython:uv
  #777·dependabot[bot] opened Aug 24, 2026Bot·

  ·

• feat: reuse local eval-agent-server images for --no-modal SWE-Bench grading

  #775·summerview1997 opened Aug 23, 2026·

  ·

• fix: alias locally-built agent-server image instead of failing on tag mismatch

  #774·summerview1997 opened Aug 23, 2026·

  ·
  1

• feat(swebench): record structured assembly cleanup telemetry

  #773·yefuyou opened Aug 20, 2026·

  ·
  1

• fix(commit0): count errored report instances

  #772·onatozmenn opened Aug 14, 2026·

  ·
  1

  2

• fix: retry errored instances on resume

  #771·onatozmenn opened Aug 14, 2026·

  ·
  1

  2

• fix(swebench): build reports after Modal evaluations

  #770·onatozmenn opened Aug 14, 2026·

  ·
  1

  2

• Add an inspectable fault-injection reliability benchmark

  #765·PranavMishra28 opened Jul 30, 2026·

  ·
  1

• build(deps): bump the version-all group across 1 directory with 6 updates

  dependenciesgithub_actions
  #762·dependabot[bot] opened Jul 27, 2026Bot·

  ·

• do not retry for max-limit or stuck-in-loop errors

  #750·yiqingxyq opened Jun 12, 2026·

  ·

• feat: per-instance intelligent model routing across all 5 default-agent benchmarks

  #742·juanmichelini opened Jun 9, 2026Collaborator·

  ·

• [codex] Add EvoClaw benchmark inference

  #705·xingyaoww opened May 7, 2026Contributor·

  ·

• [DO NOT MERGE] ci(swtbench): revert to Blacksmith runner (fallback for #495)

  #673·simonrosenberg opened Apr 20, 2026Member·

  ·
  1

• Fix GAIA build target docs: binary-minimal -> binary

  #667·simonrosenberg opened Apr 15, 2026Member·

  ·
  1

• Remove redundant 'Apply workflow_dispatch overrides' step from build workflows

  #660·simonrosenberg opened Apr 13, 2026Member·

  ·
  1

• ci: retire Blacksmith from all GitHub Actions workflows

  #649·raymyers opened Apr 9, 2026·

  ·
  1

• fix: reset BuildKit cache between retries for base/assembly builds

  #631·simonrosenberg opened Apr 4, 2026Member·

  ·

• Update Claude ACP package references

  #629·simonrosenberg opened Apr 3, 2026Member·

  ·
  3

• NeMo Evaluator Integration

  #455·simonrosenberg opened Feb 26, 2026Member·

  ·
  1

  27

  

• Add security benchmark with ASTRA

  #361·XZ-X opened Jan 26, 2026·

  ·
  11

• Agentic code search

  #141·adityasoni9998 opened Dec 8, 2025Contributor·

  ·
  5