.png)
The highest-scoring model is often the wrong default for a security harness. The billable unit is a completed workflow, and among the labs teams actually buy from, a small score improvement still costs 3 to 33 times more.
I pulled the current OpenRouter benchmark results on September 21, 2026, and compared quality with measured task cost for four jobs that resemble cybersecurity harness work, hard reasoning, tool use, research, and search. The set is limited to OpenAI, Anthropic, Google Gemini, xAI Grok, DeepSeek, and Mistral. No Mistral run won a slot on these four boards. If the expensive model does not buy a meaningful increase in completed tasks, it is a poor production choice.
At a Glance
-
Gemini 3.7 Flash reaches 94.3% on GPQA Diamond for $0.031 per question, 0.1 percentage points behind GPT-6 Astra at 94.4% and $0.099.
-
DeepSeek V4.1 Flash is the cheapest tool-use choice within five percentage points of the τ²-Bench Airline leader, at 75.7% and $0.018 per task against Gemini 3.7 Flash at 80.6% and $0.077.
-
GPT-5.6 Luna is the efficient research and search pick in this comparison, costing $0.10 on DeepSearchQA and $0.063 on WideSearch.
-
The live OpenRouter model catalog puts those value picks between $0.60 and $3.75 per million output tokens, a 6x rate-card spread before the actual task shape is included.
-
Research is where the bill explodes. Claude Opus 5 costs 33 times GPT-5.6 Luna on DeepSearchQA for four extra points.
Token prices miss the workflow bill
Model pricing pages are useful for estimating a request. They are a weak way to estimate a security harness. A harness sends tool definitions, system instructions, file contents, prior results, and repeated calls. Search tasks add a search provider and a turn budget. The model's output can be the smallest part of the operational decision.
OpenRouter's benchmark pages report the cost of the evaluated task, alongside accuracy, output tokens, timing, and the provider or search configuration. That makes them more useful for this question than a static input and output price list. The dollar figures below describe those benchmark runs, not a universal price for every prompt.
I used four benchmarks as job proxies. GPQA Diamond tests difficult questions written by subject experts. τ²-Bench Airline tests multi-turn tool use under policy constraints. DeepSearchQA measures research questions with a search loop. WideSearch measures multi-step search and structured extraction. None is a cyber range, so these results do not establish vulnerability discovery or incident response performance.
The raw GPQA table is led by Sakana Fugu Ultra, a multi-agent router that sits at 94.6% and $1.15 per question. Most security teams will never put that in a config file. This comparison drops Fugu, GLM, Kimi, MiniMax, Gemma, and Auto Router, and keeps the six labs that actually show up in production routing.
The score gap is often cheaper than the quality gap
A simple operating rule drives the rest of the analysis. For each benchmark, it selects the cheapest configuration within five percentage points of the top score among those labs. I call the supporting calculation the Cost per Correct Outcome model, and calculate it by dividing the measured cost per task by the accuracy. Treat that as a comparison tool, because a production task can still diverge from an independent benchmark question.
Source: OpenRouter benchmark pages, retrieved September 21, 2026. Comparison limited to OpenAI, Anthropic, Google Gemini, xAI Grok, DeepSeek, and Mistral. Gray bars are the highest-scoring run. Teal bars are the cheapest run within five percentage points. Axis values are ordinary dollars per completed task.
Each panel is one job. Gray is the highest score in this set, teal is the close cheaper model. The line under the panel is the decision, 3 times the cost for 0.1 extra points on hard reasoning, 4 times the cost for 4.9 extra points on tool use, 33 times the cost for 4 extra points on research, and 13 times the cost for 4 extra points on search.
I also queried OpenRouter's live model catalog, because those task costs will move when providers and routing change. The value picks have output rates from $0.60 per million tokens for DeepSeek V4.1 Flash to $3.75 for Gemini 3.7 Flash. That 6x rate-card spread is a separate finding from the public model API. The benchmark task costs still matter more for a harness because they include the actual workload shape.
Cheat sheet
Print this table. Pin it next to the harness config. Re-run the analysis when the leaderboards move.
The GPQA result is the cleanest example inside the common-lab set. GPT-6 Astra leads at 94.4% and costs $0.099 per question. Gemini 3.7 Flash scores 94.3% at $0.031, a gap of one tenth of a point and 3.2 times the cost. Under the benchmark's observed accuracy, the expected cost per correct outcome is about $0.033 for Gemini 3.7 Flash versus $0.105 for GPT-6 Astra.
That makes Gemini 3.7 Flash the default for difficult reasoning when the task can tolerate that tiny gap. A security team should still use the frontier model for a narrow class of decisions where the extra tenth of a point changes the result. The mistake is paying the frontier price for every routine analysis.
Tool use exposes a wider quality tradeoff. Gemini 3.7 Flash leads τ²-Bench Airline at 80.6% and $0.077 per task. DeepSeek V4.1 Flash reaches 75.7% for $0.018. It is 4.9 percentage points behind the leader and 4.3 times cheaper.
That is a good default for repetitive harness actions such as calling a scanner, parsing a result, checking a policy, or deciding which routine tool to call next. I would move up to Gemini 3.7 Flash when a failed tool decision has a high recovery cost. 75.7% still needs a human call about what a bad tool choice costs to unwind.
Search costs change the model ranking
DeepSearchQA and WideSearch are closer to the shape of an investigation. Their results include search providers, turn budgets, model output, and the resulting task cost. This is the part of the comparison that token pricing hides.
On DeepSearchQA, Claude Opus 5 leads the listed configurations at 77.0% and $3.32 with Parallel search. The Perplexity configuration of the same model scores 76.5% at $1.69. GPT-5.6 Luna reaches 73.0% at $0.10. Luna is four percentage points below the leader and 33 times cheaper than the Parallel Claude run. Even the cheaper Claude configuration is still about 17 times Luna.
For routine research, GPT-5.6 Luna is the sensible starting point. Claude Opus 5 earns its price when the search question is rare, expensive to verify, or likely to require a long chain of evidence. Search configuration matters as much as model choice, so a recommendation that names only the model is incomplete.
WideSearch makes the same point with a different metric. GPT-5.6 Sol with Perplexity and a 25-turn budget scores 84.0% on answer-item accuracy and 21.0% complete tables at $0.83 per question. GPT-5.6 Luna on the 5-turn Perplexity run scores 80.0% and 18.0% complete tables at $0.063. Luna is 13 times cheaper. The same Luna model at 25 turns reaches 83.4% answer-item accuracy and 22.0% complete tables for $0.17, which beats Sol on complete tables while staying well under Sol's bill.
*Source: Suzu Labs analysis of OpenRouter benchmark results retrieved September 21, 2026. Each job has its own dollar scale so a $0.031 reasoning task stays visible next to a $3.32 research task. Gray is the highest-scoring run. Teal is the value pick.*
The model recommendation changes when the success condition changes. A harness that needs every field in an extracted table should optimize complete-table accuracy. A triage harness that needs to find most relevant items quickly may prefer answer-item accuracy and a lower task cost. The benchmark does not make that decision for you.
My recommendations
Start with Gemini 3.7 Flash for difficult reasoning. It is close enough to GPT-6 Astra that measured cost dominates the decision.
Repetitive tool use should start with DeepSeek V4.1 Flash. Keep Gemini 3.7 Flash as the escalation model when an incorrect tool choice can trigger a costly retry, a bad change, or a missed control.
Research and web search should start with GPT-5.6 Luna, then test the search provider and turn budget as part of the harness. DeepSearchQA and WideSearch show that a cheaper model with a tighter budget can deliver a better operating point than a more capable model with an expensive search loop.
These are defaults, not permanent rankings. OpenRouter reports default routing when it is available, and the benchmark pages can change as models, providers, and evaluation runs change. A production team should pin the provider, record the prompt and tool budget, and replay a representative sample of its own tasks.
The practical benchmark
The final test is a security harness with a fixed task set, an error taxonomy, and a budget limit. Measure completed tasks, valid tool calls, useful findings, recovery attempts, and cost per acceptable result. A model that is 5 percentage points better on a public benchmark may still be worse for your workload if it produces longer outputs or triggers more search turns.
I would keep one expensive model in the system, but I would make it an escalation path. Let the cheaper model handle routine calls and send ambiguous, high-impact, or failed tasks upward. That design turns model selection into a control loop instead of a permanent bet on one leaderboard.
The highest score is a useful ceiling. The best security default is the model that reaches the required task quality while leaving enough budget for the next tool call.
Sources
.png)
.png)
-1.png)
.png)
.png)
