The cheapest LLM is not necessarily the cheapest way to complete a task. A model priced at $0.03 per million input tokens can still be expensive if it fails to use the right tool, produces invalid JSON, or requires another attempt with a stronger model. The practical objective is to minimize cost per completed task—not merely cost per token—and that requires routing based on completion evidence, not model price alone.
Flow AI takes that approach: it uses completion-signal data from 86,570 real agent runs, tracks whether tasks actually finish, and routes each request toward the cheapest capable model. Updated 2026-10-25, its central claim is straightforward: route for work completed per dollar, then preserve control for reproducibility and operational failures.
Start with the unit of economics: completed tasks
Most cost comparisons divide total spend by tokens or requests. That can hide the expense that matters most to an agent system: retries. A cheap first attempt that stops halfway through a workflow is not cheap if the agent must call a larger model to repair the result. Conversely, a moderately priced model that completes the task in one pass may be the better economic choice.
Flow AI’s Cortex intelligence layer measures whether an agent’s task actually finished using tool-use and completion signals rather than prompt content. It then follows three steps:
- Measure: Determine whether the model acted and completed the requested task.
- Route: Rank models by cost per completed task, subject to a completion floor. Escalate to a stronger model only when the cheaper option is insufficient.
- Adapt: Re-tune the routing as agents, tasks, prices, and quotas change.
That distinction changes the decision rule. Instead of asking, “Which model has the lowest token price?” you ask, “Which model gives me the lowest expected cost for a task that finishes?”
Flow AI reports 4.4× more agent output per dollar than defaulting to a single provider, measured across its 86,570-run benchmark. It also advertises 4.2× work / $, 119K agent runs, and 76% cheaper cost compared with the baseline. These figures are especially relevant for agent workloads because completion, tool use, and retries determine useful output—not merely generated text.
Use automatic routing for ordinary traffic, but understand its boundaries
For general request traffic, Flow AI exposes a default model named `flow-1`, which routes to the cheapest capable model for each request. It also supports the passing model `auto`, described as serving the cheapest model that can do the job. The API is OpenAI-compatible at `https://api.flowaiapi.com/v1`, and Flow AI is compatible with OpenAI- or Anthropic-compatible agent harnesses through a single base URL.
Automatic routing is not a black box that simply sorts a price list. Flow AI first classifies tasks with inexpensive, non-LLM heuristics such as request length, code, tool or JSON schema requirements, and multimodal parts. Routing decisions are cached by system prompt, which avoids repeating classification work on every turn in an otherwise similar conversation.
The cascade mechanism provides a practical safety net. If a provider errors, Flow AI fails over to the next capable provider. The response includes a `_flowaiapi.cascade` array showing each model tier attempted before success, and rejected cheap attempts in cascade mode are not charged to the user. In other words, the system can try a low-cost lane, reject an inadequate attempt, and escalate without billing the customer for the failed attempt.
This is materially different from routing only for availability. Failover protects the request from an outage. Task-aware routing tries to protect the outcome. A provider can be healthy and technically responsive while still being the wrong model for the work.
Treat price as a range, not a permanent sticker
Model prices on Flow AI float within a floor-to-ceiling band. The floor is the minimum supplier price; the ceiling is the published API rate. Prices adjust continuously with supply and demand. Heavy demand raises prices incrementally—cent at a time—to spread load toward alternatives, while Slack demand pulls prices back toward the floor.
This makes pricing dynamic in two ways. First, the same model can have a different effective price at different moments. Second, the cheapest model for one task may change without any change to your code.
The examples show why token price alone is insufficient. `minimax-m2.7` has a clearing price of $0.03 input / $0.10 output, against a published rate of $0.25 / $1.00, representing 90% savings. By contrast, `claude-opus-4.8` clears at $15.00 input / $75.00 output with no savings. These are not interchangeable models for every job; they demonstrate why a router needs a capability and completion signal alongside a live price.
The market currently reports 76% average savings below published API rates, 55 live models, 68.7 billion tokens cleared, and $12,564 total network savings. Models are ranked by lifetime token volume, giving operational context to demand and reliability. A downward arrow next to a model indicates that its price is currently getting cheaper, which can matter when the router recalculates the cheapest capable path.
Pin models when reproducibility matters
Automatic routing is ideal for variable production workloads, but it creates a testing problem: if each request may select a different model, an experiment cannot reliably attribute a change in quality to a model change. Flow AI therefore supports pinned routes.
Use `pin:<model>` or the `X-FlowAI-Route: pinned` header to force a specific model. Most models on the platform are marked as pinnable. Pinned requests bypass all response caching, ensuring that each call reaches the model directly for reproducible experiments. If the pinned model cannot serve a request, the API returns a 503 response with `model_unavailable`.
That behavior is intentional. A pinned route is an experiment constraint, not a promise that the provider will always be available. For a controlled benchmark, a clear failure is preferable to silently substituting another model. For production, automatic routing with failover is generally the more resilient choice.
The broader model catalog also exposes practical distinctions. Models are classified as either pinnable or self-host. Self-host examples include `qwen3-8b`, `qwen3-14b`, `qwen3-32b`, `mistral-small`, `qwen2.5-7b`, `llama3.3-70b`, `qwen2.5-coder` variants, `devstral`, `gemma3-4b`, `phi-4-mini`, and `gemma2-2b`. Several models—including `nemotron-3-ultra` and some provider-prefixed free routes—are offered at $0.00/$0.00, but free does not mean universally capable. The router must still decide whether a free model is appropriate for the task.
Use panels to test alternatives without building a routing lab
When model selection is uncertain, a panel can answer one prompt with several models in parallel. Flow AI’s `/v1/panel` endpoint supports up to 10 models answering in one call, with per-leg cost and latency. It is opt-in per key in the dashboard and off by default.
A panel is useful for three specific jobs:
1. Selecting a default: Compare cheap candidates on real task completion rather than generic benchmark scores.
2. Building a jury: 4seen AI uses multi-model juries with four model families voting in one call.
3. Separating cost from quality: Inspect each leg’s latency and price before deciding which model should handle the task.
Panel calls should not be treated as free information. If ten models answer the same prompt, you are paying for ten outputs. The economic value comes from using the panel to create a better production route, then switching that route to `flow-1` or `auto` where appropriate.
For web-grounded work, Flow AI provides search-native lanes including Gemini, Perplexity Sonar, and GPT web search, with citations normalized into one format. This is another case where “cheapest model” is too narrow: a model without the required grounding capability cannot complete the task at any price.
Add operational controls before optimizing further
A router that works only under ideal conditions is not production infrastructure. Flow AI includes per-key spend caps, lane-health monitoring every six hours, and automatic failover when a provider degrades. Cost attribution is available per agent and per task type, showing which agents finish work efficiently and which models earn their price.
That attribution closes the measurement loop. You can distinguish a low token price from low task cost, identify tasks that consistently require escalation, and determine whether an expensive model is justified by its completion rate. Paperclip, an autonomous agent fleet, has completed 118,776 runs fully managed by Flow AI; Flow AI itself is dogfooded by a fleet of 60+ autonomous agents running production workloads daily.
The platform also has a community supply layer called Hive. Contributors can run models from a Mac or VPS through a Flow AI node and share permitted spare capacity, earning credits based on reliability and demand. This helps explain the supply-sensitive price model, but it also introduces a reliability consideration: community capacity should be judged by observed service behavior, not merely by its advertised cost.
A practical routing policy
A sound implementation has different policies for different request classes. Use `flow-1` or `auto` for high-volume production traffic where completion matters more than fixed model identity. Use pinned routes for regression tests, debugging, and benchmark comparisons, accepting the 503 failure mode when the selected model is unavailable. Use panels sparingly to evaluate candidates or construct juries. Route web-grounded tasks only to lanes with the required search capability. Keep spend caps and per-agent cost attribution enabled from the beginning.
Also account for the billing layer. Flow AI publishes model cost pass-through plus a flat 2.5% spread: `buyer_charge_usd` equals `cost_usd × 1.025`. The response metadata exposes that calculation, so teams can separate the underlying model cost from the buyer charge. New accounts receive a 7-day free trial; afterward, an active membership costs $4.99/month or $45/year. The API base URL is `https://api.flowaiapi.com/v1`, and `GET /v1/models` lists currently available models.
For image inputs, the gateway auto-fetches remote HTTP(S) URLs with an 8-second timeout and 5MB file-size cap. Failed image URL fetches return a clear 400 error. These are useful boundaries to design around rather than discover during an incident.
The bottom line
The right way to route LLM requests is not to choose the lowest sticker price. It is to maintain a live ranking of models by cost per completed task, enforce a completion floor, and escalate only when necessary. Flow AI implements that model through Cortex signals, dynamic pricing, cascades, provider failover, and explicit controls for pinning and panels.
For most production agent workloads, the winning default is `flow-1` or `auto`: cheap first, measured by whether the work actually finishes, with stronger models reserved for the tasks that require them. Pin models when reproducibility is the priority; use panels when you need evidence to choose the next route. The objective is not the cheapest token. It is the cheapest completed outcome.