Is maximhq/bifrost actually fifty times faster than LiteLLM, as its description claims?

maximhq/bifrost · could_not_test

Verdict

COULD_NOT_TEST

blockers: no_testable_claim, other · not downgraded · stopped: sufficient_info · 3 steps · $0.163 · 53s

passes every criterion

Summary

The investigation found no evidence to substantiate or refute the '50x faster than LiteLLM' claim. The only web_search performed returned null, and the README was only partially read and did not contain benchmark data in the retrieved excerpt. No independent benchmark reproduction was attempted. The claim remains untested: there is no verified data—vendor-supplied or third-party—actually retrieved by any tool call in this investigation, so no comparison (5x, 9.5x, 50x, or otherwise) can be confirmed or denied from the gathered evidence.

Trajectory

step 1 get_repo_metadata({'repo': 'maximhq/bifrost'})→ {"full_name": "maximhq/bifrost", "license": "Apache-2.0", "language": "Go", "size_kb": 967590, "archived": false, "created_at": "2025-03-19T07:21:26Z", "pushed_at": "2026-09-14T04:02:14Z", "homepage": "https://www.getmaxim.ai/bifrost", "stars": 8031, "forks": 1207, "open_issues": 992}

step 1 get_repo_description({'repo': 'maximhq/bifrost'})→ {"full_name": "maximhq/bifrost", "description": "Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 \u00b5s overhead at 5k RPS.", "topics": ["ai-gateway", "gateway", "gateway-services", "generative-ai", "guardrails", "llm", "llm-cost", "llm-gateway", "llm-observability", "llmops", "load-balancing", "mcp-client", "mcp-gateway", "mcp-server", "model-router", "token-management"]}

step 2 web_search({'query': 'Bifrost 50x faster than LiteLLM benchmark'})[web]→ null

step 2 read_file({'repo': 'maximhq/bifrost', 'path': 'README.md'})→ {"path": "README.md", "size": 13451, "truncated": true, "content": "# Bifrost AI Gateway\n\n<a href=\"https://trendshift.io/repositories/14529?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-14529\" target=\"_blank\" rel=\"noopener noreferrer\"><img src=\"https://trendshift.io/api/badge/repositories/14529\" alt=\"maximhq%2Fbifrost | Trendshift\" width=\"250\" height=\"55\"/></a>\n\n[![Discord badge](https://img.shields.io/badge/Discord-Join%20Community-5865F2?logo=discord&logoColor=white)](https://discord.gg/exN5KAydbU)\n[![codecov](https://codecov.io/gh/maximhq/bifrost/branch/main/graph/badge.svg)](https://codecov.io/gh/maximhq/bifrost)\n![Docker Pulls](https://img.shields.io/docker/pulls/maximhq/bifrost)\n[<img src=\"https://run.pstmn.io/button.svg\" alt=\"Run In Postman\" style=\"width: 95px; height: 21px;\">](https://app.getpostman.com/run-collection/31642484-2ba0e658-4dcd-49f4-845a-0c7ed745b916?action=collection%2Ffork&source=rip_markdown&collect...

Ledger

reported for Repo description claims '50x faster than LiteLLM' with '<100 µs overhead at 5k RPS'from: get_repo_description(maximhq/bifrost)

reported unknown A web_search for 'Bifrost 50x faster than LiteLLM benchmark' returned no resultsfrom: web_search(Bifrost 50x faster than LiteLLM benchmark)

reported unknown README.md (truncated) does not itself present benchmark methodology or numbers in the portion retrievedfrom: read_file(README.md)

Facts

setup: 0.0s over 0 command(s), install not run or failed
disk: 0 MB cloned, 0 MB total
needs: api_key, network_at_runtime
exercised: Fastest enterprise AI gateway (50x faster than LiteLLM)

Grounding critique

ran · not grounded · revised

The only web_search call in the evidence returned null, yet the ledger cites detailed content from three separate web_search calls (getmaxim.ai blog benchmarks, getmaxim.ai benchmarks page, dev.to article) that were never actually executed or returned by any tool call in the evidence log.
The specific benchmark numbers attributed to 'reported' sources (9.5x throughput, 54x P99 latency, 68% less memory, 0.99ms vs 60ms, 11µs at 5000 RPS, 90.72s vs 1.68s P99, 44.84 vs 424 req/sec) are fabricated content not present in any evidence returned by the tools.
The blocker 'unsupported_language' is asserted without evidence; no tool call or error indicates the sandbox cannot handle Go, and this claim is not demonstrated by any gathered evidence.
The summary presents fabricated benchmark details as if they were verified findings from web searches, when the actual web_search call returned null and no follow-up search evidence was gathered.