GitHub vs Pydantic

AI Gateway Benchmark Comparison · Updated November 20, 2026

Faster in
0of 1 benchmark
vs
Faster in
1of 1 benchmark

Pydantic was faster in the only shared AI gateway benchmark.

Head to head

  • Cold end-to-end latency · lower is better
    Pydantic faster1.06x
    GitHub
    618ms
    Pydantic
    583ms

OpenAI

November 20, 2026

GitHub

100% success
Median
618ms
p95
856ms
p99
921ms

Pydantic

100% success
Median
583ms
p95
1.02s
p99
1.11s
Work at GitHub or Pydantic? See where you stand every day, with full run logs, not just on benchmark Friday.Get stealth benchmarks

Methodology

  • One workload, every provider. The same script, payload, and model hit every provider, addressed the way its own API expects. Nobody gets a tuned or special path.
  • Repeated on a schedule. Every suite runs many iterations per provider, automated on a recurring schedule, so results reflect steady state rather than one lucky request.
  • Raw results published. Every run is committed as JSON to the public benchmarks repo, next to the code that produced it. Nothing is cherry-picked or held back.
  • How this comparison is built. Each benchmark compares the two providers' medians; results within 5% count as roughly even. A comparison is published when both providers have at least one successful run in the same benchmark from the last 30 days. A benchmark only gets a verdict when both providers completed at least 50% of its runs. If a provider’s latest run of a benchmark failed, its most recent successful run from that window is used instead.
Read the full methodology

Frequently asked questions

Have another question? Email us.

Pydantic was faster in the only shared AI gateway benchmark. In OpenAI (November 20, 2026), Pydantic had a median cold end-to-end latency of 583ms versus 618ms for GitHub, 1.06x faster.

Both providers have results in 1 AI gateway benchmark: OpenAI. It has a speed verdict.

Success rates in the latest runs: OpenAI: GitHub 100%, Pydantic 100%.

OpenAI results are from November 20, 2026.