Methodology
What We Measure
Each run executes a cold clone + install + typecheck cycle of the opencode repository inside a fresh sandbox: install system packages and Node.js, download and unpack Bun, shallow-clone opencode at a pinned commit, run bun install, then bun typecheck.
Each provider runs 1 iteration per scheduled benchmark run. Benchmarks run via GitHub Actions and results are committed to the public benchmarks repo.
How We Score
The Composite Score is the median number of the 7 benchmark phases (prepare, cache clear, bun download, bun unpack, clone, install, typecheck) a provider completed before failing, if any — shown as a fraction like 7/7.
The other metrics (Total, Prepare, Clone, Install, Typecheck) are real-world phase durations in seconds — providers are ranked by median duration (lower is better) when one of those is selected. The benchmark requires curl, root/sudo, and a package manager (apt, dnf, or apk) inside the sandbox — providers without those capabilities fail early, which shows up as a low phase count.