OpenRouter vs PydanticAI Gateway Benchmark Comparison
Pydantic was faster in 1 of 2 shared AI gateway benchmarks, and 1 was roughly even.
- Shared benchmarks
- 2
- Last run
- October 30, 2026
Scorecard
Median of each benchmark's primary metric; lower is better. Medians within 5% count as roughly even, and a benchmark gets no speed verdict when either provider completed fewer than 50% of runs.
Anthropic
Run October 30, 2026Time for a cold request routed through the gateway to Claude Haiku 4.5 to return its complete response.
Cold end-to-end latency
Pydantic had a 1.19x lower median cold end-to-end latency (393ms vs 469ms for OpenRouter).
Median cold end-to-end latency over time
15 runs from July 24, 2026 to October 30, 2026.
OpenAI
Run October 30, 2026Time for a cold request routed through the gateway to GPT-5.4 Mini to return its complete response.
Cold end-to-end latency
The medians are within 5% of each other (698ms vs 681ms), so this counts as roughly even.
Median cold end-to-end latency over time
11 runs from August 21, 2026 to October 30, 2026.
Methodology
- One workload, every provider. The same script, payload, and model hit every provider, addressed the way its own API expects. Nobody gets a tuned or special path.
- Repeated on a schedule. Every suite runs many iterations per provider, automated on a recurring schedule, so results reflect steady state rather than one lucky request.
- Raw results published. Every run is committed as JSON to the public benchmarks repo, next to the code that produced it. Nothing is cherry-picked or held back.
- How this comparison is built. Each benchmark compares the two providers' medians; results within 5% count as roughly even. A comparison is published only when both providers completed the latency benchmark on at least two of the same model families reliably: at least 50% success for each provider, in benchmark runs from the last 30 days.
Frequently asked questions
Have another question? Email us.
Pydantic was faster in 1 of 2 shared AI gateway benchmarks, and 1 was roughly even. In Anthropic (October 30, 2026), Pydantic had a median cold end-to-end latency of 393ms versus 469ms for OpenRouter, 1.19x faster. In OpenAI (October 30, 2026), OpenRouter had a median cold end-to-end latency of 698ms and Pydantic 681ms, which counts as roughly even.