Boat vs CreateOS

Sandbox Benchmark Comparison · Updated November 6, 2026

CreateOS was faster in the only sandbox benchmark with a speed verdict.

Head to head

  • Time to interactive · lower is better
    No verdict
    Boat
    12.83s48%
    CreateOS
    332ms80%
  • Total workload time · lower is better
    CreateOS faster1.18x
    Boat
    62.23s
    CreateOS
    52.68s

Boat

48% success · November 20, 2026
Median
12.83s
p95
13.90s
p99
14.04s

CreateOS

80% success · November 6, 2026
Median
332ms
p95
394ms
p99
417ms

Boat

100% success · November 20, 2026
Median
62.23s

CreateOS

100% success · November 6, 2026
Median
52.68s
Work at Boat or CreateOS? See where you stand every day, with full run logs, not just on benchmark Friday.Get stealth benchmarks

Methodology

  • One workload, every provider. The same script, payload, and model hit every provider, addressed the way its own API expects. Nobody gets a tuned or special path.
  • Repeated on a schedule. Every suite runs many iterations per provider, automated on a recurring schedule, so results reflect steady state rather than one lucky request.
  • Raw results published. Every run is committed as JSON to the public benchmarks repo, next to the code that produced it. Nothing is cherry-picked or held back.
  • How this comparison is built. Each benchmark compares the two providers' medians; results within 5% count as roughly even. A comparison is published when both providers have at least one successful run in the same benchmark from the last 30 days. A benchmark only gets a verdict when both providers completed at least 50% of its runs. If a provider’s latest run of a benchmark failed, its most recent successful run from that window is used instead.
Read the full methodology

Frequently asked questions

Have another question? Email us.

CreateOS was faster in the only sandbox benchmark with a speed verdict. In DAX (November 6, 2026), CreateOS had a median total workload time of 52.68s versus 62.23s for Boat, 1.18x faster.

Both providers have results in 2 sandbox benchmarks: Burst TTI and DAX. 1 of them has a speed verdict.

Success rates in the latest runs: Burst TTI: Boat 48%, CreateOS 80%; DAX: Boat 100%, CreateOS 100%. Burst TTI has no speed verdict because Boat completed fewer than half of its runs; CreateOS's median time to interactive was 332ms.

Burst TTI and DAX results are from November 6, 2026.