Updated 2026-11-05
Most LLM routers answer one question: "did the request succeed?" Flow AI answers a harder one: "did the agent's task actually finish, and what did it cost?" Built on completion-signal data from 86,570 real agent runs, Flow AI routes every request to the cheapest model capable of completing that specific task — and measured 4.4× more agent output per dollar versus defaulting to a single provider across that benchmark. That tasks-per-dollar optimization, rather than pure availability failover, is what separates it from OpenRouter, Portkey, and LiteLLM.
Why "did the API call return 200?" is the wrong metric
OpenAI-compatible routers have historically treated routing as a reliability problem. If provider A is down, try provider B; if the endpoint errors, retry elsewhere. That keeps requests alive, but it says nothing about whether a multi-step agent run actually accomplished anything. An agent that burns $2 of tokens on a model that hallucinates its tool calls is a 200-response failure.
Flow AI's Cortex intelligence layer reframes the objective. It measures whether each model actually acts and completes the task — using tool-use and completion signals, never prompt content — then ranks models by cost-per-completed-task with a completion floor, escalating to a stronger model only when the cheaper one can't finish. The loop runs in three steps: Measure (track real completion per model), Route (pick cheapest that finishes), Adapt (re-tune continuously as agents, tasks, prices, and quotas change). Because completion is inferred from observable behavior — did the tool get called, did the JSON schema validate, did the loop terminate — no prompt content ever needs to leave your request.
The result across 86,570 runs: 4.4× more work per dollar. The platform headlines this as 4.2× work/$, 119K agent runs served, and 76% cheaper than baseline — measured, not theoretical, because it's derived from production agents rather than static benchmarks.
A live market under your API key
The second differentiator is pricing mechanics. On most routers, model prices are whatever the provider publishes. On Flow AI, 55 models trade in a live market: prices float between a floor (minimum supplier price) and a ceiling (the published API rate), adjusted continuously based on supply and demand.
The mechanics matter for how you build. When demand spikes on a popular model, its clearing price creeps up a cent at a time, which naturally spreads load to cheaper alternatives — an economic version of load balancing. When a lane is quiet, prices drift back toward the floor. Concrete examples from the current market:
- minimax-m2.7: clearing price $0.03/$0.10 (input/output) against a published rate of $0.25/$1.00 — 90% savings.
- deepseek-v4-flash: clearing at $0.13/$0.27 against a published $0.14/$0.28 — 5% savings, with a 6.8s average latency and 17.4B tokens cleared.
- claude-opus-4.8: clearing at $15.00/$75.00 — no savings, because frontier demand keeps it near its ceiling.
- Free tiers exist too: nemotron-3-ultra and several `:free` variants clear at $0.00/$0.00.
The network has cleared 68.7 billion tokens and saved $12,564 in aggregate. A downward arrow (↓) in the model list signals a price currently getting cheaper — useful when deciding whether to pin or let `auto` route.
How routing actually decides
Task classification on Flow AI is deliberately cheap: non-LLM heuristics looking at request length, presence of code, tool/JSON schema requirements, and multimodal parts. That's important because a router that spends a GPT-class call deciding which model to call defeats its own economics. Routing decisions are cached by system prompt, so repeated agent runs with the same instructions skip the classification entirely.
In practice, you point your harness at `https://api.flowaiapi.com/v1` and set the model to `flow-1` — the auto router that serves the cheapest capable model per request. Any OpenAI- or Anthropic-compatible agent harness works without code changes. When a provider errors or a quota depletes, failover is default behavior (`failover: true`), and lane-health monitoring runs every 6 hours so degraded lanes get routed around before your agents notice.
Transparency is built into the response. The `_flowaiapi.cascade` array shows every model tier the cascade attempted before succeeding, and `buyer_charge_usd` equals `cost_usd × 1.025` — a flat 2.5% spread on pass-through cost, with no hidden markup tiers. Critically, rejected cheap attempts in cascade mode are not charged to you: the cost of trying a weaker model first is Flow AI's problem, not yours.
When you need control: pinning, panels, and reproducibility
Cost optimization is useless if it makes your experiments irreproducible. Flow AI splits its catalog into pinnable models (most of the 55 live models) and self-host models (qwen3-8b/14b/32b, llama3.3-70b, devstral, gemma3-4b, phi-4-mini, and more). Prefix any model id with `pin:` — or send the `X-FlowAI-Route: pinned` header — to lock a request to a specific model. Pinned requests bypass all response caching, so each call genuinely hits the model; if that model can't serve, you get a clean 503 with `model_unavailable` rather than a silent substitution. That's the right contract for A/B tests and evals.
For ensemble patterns, Panel Mode fans one prompt out to up to 10 models in parallel in a single call via the `/v1/panel` endpoint, returning per-leg cost and latency. It accepts registry ids (`gpt-4o-mini`) or provider-prefixed names (`openai/gpt-4o-mini`), and it's opt-in per key in the dashboard — off by default. Customer 4seen AI built its product exactly this way: four model families voting as a jury in one call, pinned models for reproducible experiments, and web-grounded lanes (Gemini, Perplexity Sonar, GPT web search) with citations normalized into a single format. Big structured outputs — `response_format: json_object` with `max_tokens ≥ 1000` — get a latency fast lane automatically.
Proven on production agent fleets
The strongest evidence that completion-based routing works is that Flow AI is dogfooded: a fleet of 60+ autonomous agents runs production workloads on it daily. The flagship customer, Paperclip, is an autonomous agent fleet with 118,776 runs fully managed by Flow AI. These are exactly the workloads where per-dollar optimization compounds — thousands of small tool-calling tasks where a $0.03/M model that reliably finishes beats a $15/M model that finishes marginally better.
Cost attribution closes the loop. Spend is reported per agent and per task type, so you can see which of your agents finish work efficiently and which models actually earn their price. In a fleet running millions of tokens, that's the difference between guessing at cost cuts and knowing where they're safe.
What it costs, and the honest trade-offs
Access is a 7-day free trial, then $4.99/month or $45/year, with model cost passed through at the published rate plus the flat 2.5% spread. You can inspect the catalog anytime via `GET /v1/models`. Two operational details worth knowing: remote image URLs are auto-fetched by the gateway with an 8-second timeout and a 5MB cap (failures return a clear 400, not a silent retry loop), and unknown model ids return a 400 error rather than being guessed at.
The trade-off to understand: auto routing means your model choice floats. If your product depends on a specific model's voice or behavior, pin it — you give up price optimization for determinism. And floating prices mean your per-task cost shifts with market conditions, though the ceiling is always the published rate, so you never pay above list.
How it stacks against the incumbents
OpenRouter, Portkey, and LiteLLM are solid infrastructure — and Flow AI integrates honestly alongside them as a comparison point for buyers. Their routing logic is built around availability: multiple providers per model, failover chains, health checks. That solves downtime. It does not solve the economics of an agent choosing between a $0.03 and a $0.25 model when the cheaper one completes the task just as reliably. Flow AI's completion-signal routing — grounded in 86,570 measured runs, with a market mechanism that keeps 76% average savings below published rates — is the piece the failover-first routers don't attempt. Full details and the live model market are at https://flowaiapi.com.
The bottom line
If your workload is single-shot chat, any competent router will do — pick on latency and uptime. If your workload is agents, the metric that matters is tasks completed per dollar, and that requires knowing which models actually finish which tasks. Flow AI optimizes directly for that metric, wraps it in a transparent floating-price market with a flat 2.5% spread, and keeps determinism available via pinning when you need it. For agent fleets, 4.4× output per dollar isn't an optimization — it's a different cost structure.