Skip to content

AIAI Tools & Models

AI Model Benchmark: How AI Text Models Perform on 14 Jobs

Six configurations got full marks, one of them for about a tenth of a cent per solved hard test. This is every score, with what each model cost and where it slipped.

The logos of seven AI labs in frosted glass, standing in a row.

Updated 18 November 2026: we added GPT-6.2 Sol and GPT-6.2 Astra, which went through the same 14 jobs on 17 November.

Every AI launch arrives with a benchmark chart on which the new model wins, usually on a test the lab picked and at a setting most people will never use. Plenty of "best AI model" roundups then paraphrase those charts back to you without asking what any of it cost. We don't trust promises we can't check, so we built our own benchmark from the kind of work we hand AI each week and ran every text model in our ranking through it.

Between 12 and 17 November, 49 text model configurations got the same 14 jobs: nine hard tests in three versions each, ten everyday questions and four real work tasks. They ranged from GPT-6 Astra and Claude Opus 5.5 down to models that cost a fraction of a cent per answer. We paid for every request, kept the wrong answers along with the right ones and read the explanations behind the passing results, because a model can clear an automatic check and still tell you something false.

Quick Answer

Six configurations got full marks: GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, Gemini 4 Carbon, Grok 4.6 and Meta's small Muse Ember, which got there for about a tenth of a cent per solved hard test. Price decided far less than you'd expect, since GPT-6 Luna, Claude Haiku 5.5 and DeepSeek V4.1 Flash 1017 all scored between 95 and 98 for well under a cent per solved test. The most expensive configuration per solved test, Mistral Medium 3.5, scored 78.6, several Claude models lost points to a single rule about tool calls, and Claude Haiku 4.5 and Nemotron 3.5 Lightning propped up the table on 44 and 7. OpenAI's new GPT-6.2 Sol and GPT-6.2 Astra came close behind on 99.2 and 97.6. With so many strong models bunched near 100, the test can't put the leaders in order, so our Best AI Models ranking uses each score as a ceiling on that model's tier and leaves the order to our daily work.

Which model for what in our test

These picks come from the test results alone, so they leave out what our ranking adds on top, such as writing quality across long sessions and how a model copes with work bigger than a test.

If you wantOur pick from the testWhy
Full marks for the least moneyMuse Ember100 at $0.0011 per solved hard test, with the fastest median time, 9 seconds
The lowest cost per solved hard testGPT-6 Luna$0.0009, although it scored 95.2
Full marks from a frontier modelGPT-6.1 Sol100 at $0.009, a fifth of what GPT-6 Astra spent for the same score
The highest score from AnthropicClaude Fable 5.199.2, at $0.081 per solved hard test, the highest cost among the top 10 scores
A cheap model for real workDeepSeek V4.1 Flash 101797.6 and all four real jobs done well, at $0.0027 per solved hard test

The 14 jobs

Each of the 14 jobs is worth three points, so the ten everyday questions together count as much as one hard test or one real job, and every company, product and price inside them is invented. The hard tests come in three versions with different data, which stops a lucky guess from carrying a whole test, and the last column shows how many configurations got full marks on each job.

JobWhat the model has to doFull marks
Hard test 1: time zonesConvert publishing times between cities, including the hour that happens twice and the hour that never happens when clocks change31 of 49
Hard test 2: paginationRepair a database paging function so rows with the same timestamp are never skipped or repeated, without the shortcuts the brief rules out28 of 49
Hard test 3: a small fixFix one function in a 60-line file while changing no more than four lines and leaving the rest as it was35 of 49
Hard test 4: schedulingFit six articles into six weekday slots under eight rules, then face a set of rules that can't all be met and name the ones that clash30 of 49
Hard test 5: changing an answerA two-turn billing or editing task where the second message brings a correction, and the model has to tell a valid one from a wrong one39 of 49
Hard test 6: a planted instructionWork out a product's real specification from conflicting sources, one of which tells the AI to recommend something else25 of 49
Hard test 7: refund policyApply seven ordered rules, with their exceptions, to customer refund requests42 of 49
Hard test 8: table editsApply 15 moves, inserts, renames, undos and redos to a ranking table33 of 49
Hard test 9: toolsRepair a small three-file project with four tools, one tool call per reply and 12 calls in total29 of 49
Everyday questionsTen plain questions about dates, unit prices, return deadlines, time zones and VAT, two of which can't be answered from what's given48 of 49
Real job: auditCheck two cost reports against a request log, then handle a regrade and a colleague's suggestion to drop an inconvenient charge41 of 49
Real job: payment fixRepair a payment handler that can charge a customer twice, without creating a refund bug in the process44 of 48
Real job: SQL migrationFix a migration that attaches invoices to the wrong accounts and explain exactly which rows it damaged37 of 49
Real job: editorial handoverWrite and revise a short article over three turns, the last one in a fresh chat with only the model's own files27 of 48

The editorial handover caught out more models than any other job, and nine configurations scored nothing on it through a mix of answers cut off at the limit, missing files and handovers that described their own work wrongly. Among the hard tests, the planted-instruction audit produced the fewest full marks, while hard test 9 had the lowest average once partial scores count.

Full results

The TGK Score is the share of the 42 available points a model earned, from Good (3 points) to Fail (0) on each job, and the grading is explained in full under "How the score works" further down. Cost per solved hard test, or "per solve" below, is what a model spent on the 27 hard test versions, repairs included, divided by the number it solved. The median time is per hard test, and the real-jobs column shows points out of 12, three for each job.

RankModelTGK ScoreHard tests solvedReal jobs (of 12 points)Cost per solved hard testMedian time per hard test
1Muse Ember10027 of 2712$0.00119 s
2GPT-6.1 Terra10027 of 2712$0.005316 s
3GPT-6.1 Sol10027 of 2712$0.009015 s
4Gemini 4 Carbon10027 of 2712$0.02744 s
5Grok 4.610027 of 2712$0.03253 s
6GPT-6 Astra10027 of 2712$0.04614 s
7GPT-6.2 Sol99.227 of 2712$0.008613 s
8Muse Wonder99.227 of 2712$0.04420 s
9Claude Fable 5.199.227 of 2712$0.08122 s
10DeepSeek V4.1 Flash 101797.626 of 2712$0.002737 s
11Gemini 3.8 Flash97.627 of 2711$0.01635 s
12GPT-6.2 Astra97.627 of 2711$0.02912 s
13Claude Opus 5.597.626 of 2712$0.03416 s
14Gemini 3.7 Flash96.827 of 2711$0.01734 s
15Muse Spark 1.396.827 of 2711$0.02136 s
16Claude Haiku 5.596.026 of 2712$0.001520 s
17Gemini 4 Argon96.027 of 2712$0.01324 s
18GPT-5.6 Sol96.025 of 2712$0.02718 s
19GPT-6 Luna95.225 of 2712$0.000917 s
20Grok 4.595.227 of 2710$0.02337 s
21Gemini 4 Hydrogen95.227 of 2710$0.02720 s
22Claude Sonnet 5.593.725 of 2712$0.01718 s
23Claude Fable 5.292.924 of 2712$0.06824 s
24Hy-392.124 of 2711$0.0052141 s
25Qwen 3.8 Max 090292.124 of 2712$0.02956 s
26Claude Opus 590.523 of 2712$0.04417 s
27GPT-5.6 Luna89.723 of 2712$0.002520 s
28Hy489.726 of 279$0.019126 s
29Claude Sonnet 588.923 of 2712$0.03422 s
30Grok 4.788.925 of 278$0.04647 s
31Kimi K388.123 of 2711$0.09285 s
32GPT-6 Sol87.324 of 2710$0.009210 s
33GPT-5.6 Terra87.322 of 2712$0.01818 s
34GLM-5.387.325 of 279$0.03653 s
35Muse Spark 1.286.527 of 277$0.01827 s
36Claude Fable 585.724 of 2710$0.08617 s
37Muse Glimmer82.520 of 2710$0.006633 s
38GLM-5.3 Flash81.722 of 278$0.005139 s
39Inkling81.723 of 279$0.02035 s
40Mistral Medium 3.578.620 of 2710$0.17969 s
41Qwen 3.8 27B77.820 of 2710$0.02678 s
42MiMo V2.5 Pro75.426 of 274$0.005122 s
43Nemotron 3 Ultra73.820 of 278$0.02132 s
44MiniMax-M369.822 of 276$0.01739 s
45Gemini 3.6 Flash69.820 of 278$0.04745 s
46DeepSeek V4.1 Flash (older snapshot, third-party host)63.917 of 2710$0.008737 s
47Gemini 3.5 Flash61.114 of 279$0.17153 s
48Claude Haiku 4.544.412 of 273$0.03822 s
49Nemotron 3.5 Lightning7.10 of 271none solved19 s

Qwen 3.8 Flash-Next and K2 Horizon weren't available to us when we ran the tests, and Kimi K3.1 Pro returned nothing but rate-limit errors. Hy-4 Preview gave way to its finished release, Hy4, so those four have no score of their own.

How each family did

OpenAI

OpenAI's line-up barely tripped over anything, which makes for a dull paragraph and is exactly what you want from a model doing real work. GPT-6.1 Sol, GPT-6.1 Terra and GPT-6 Astra all got full marks, and Terra did it for about half a cent per solve. GPT-6 Astra needed five times GPT-6.1 Sol's spend for the same 100, $0.046 against $0.009, so on work like this the cheaper model is the better buy. The older models are where OpenAI slipped: GPT-6 Sol broke the tool-call rule on all three versions of hard test 9 and left one damaged invoice out of an otherwise exact SQL explanation, which put it at 87.3. GPT-6 Luna was the cheapest configuration of all 49 at $0.0009 per solve, and it lost points only on hard tests, with full marks on every real job.

The GPT-6.2 pair arrived on 17 November and went through all 14 jobs that day. GPT-6.2 Sol scored 99.2 for $0.0086 per solve, slightly less than GPT-6.1 Sol spent, and its only lost points came from the repair it needed on one version of the planted-instruction test. GPT-6.2 Astra solved all 27 hard test versions on the first attempt for $0.029 per solve, well under GPT-6 Astra's $0.046, but finished on 97.6 because the CMS description in its editorial job ran to 157 characters against a limit of 155 in two of the three turns.

Anthropic

Claude's results look worse than our daily experience of the models, and almost all of the gap comes from one rule. Hard test 9 allows one tool call per reply, and Anthropic's models broke it 15 times, more than every other family put together, which is why Fable 5.2 scores 92.9 and Opus 5 90.5. Away from that test the family was very solid, and every Claude model except Fable 5 and Haiku 4.5 did all four real jobs well. Fable 5.1 reached 99.2, and Opus 5.5 missed one version of the tool test and nothing else, for 97.6 at $0.034 per solve.

Haiku 5.5 was the family's cheap surprise at 96.0 for $0.0015 per solve, the lowest cost of any Claude model and a long way above Haiku 4.5's 44.4. Fable 5 lost points on the editorial job, where its handover claimed a 350 to 500 word limit the article had missed, and at $0.086 per solve it was also one of the priciest models in the test.

Google

Google's Flash models improved with every generation we tested, climbing from 61.1 for Gemini 3.5 Flash and 69.8 for 3.6 Flash to 96.8 for 3.7 Flash and 97.6 for 3.8 Flash, a climb that moved the last two up into B in our ranking. At the top, Gemini 4 Carbon scored 100 for $0.027 per solve, and Carbon, Argon and Hydrogen each solved all 27 hard test versions. Argon needed repairs on five of them, though, and Hydrogen lost points on the editorial job by handing back a CMS file that wasn't valid JSON.

Meta

Muse Ember, Meta's smallest model, embarrassed most of the expensive models in the test by getting full marks for $0.0011 per solve, with the fastest median time of all 49 at 9 seconds. Muse Wonder came close behind at 99.2 after a single repair, Spark 1.3 reached 96.8 and the 30B Muse Glimmer finished at 82.5. Spark 1.2 solved every hard test and still finished on 86.5, because its SQL explanation made a false claim about the original migration and its editorial handover misreported the length of its own CMS description.

xAI

Grok 4.6 scored 100 and Grok 4.5 95.2, which left the newest of the three, Grok 4.7, as the odd one out at 88.9. Its first audit answer, in full, was a promise: "I will calculate the metrics from the packet only, then check both reports against those figures." The rerun with four times the output room failed as well, and its editorial handover misreported the length of its own opening.

Chinese labs

Results from the Chinese labs ranged widely, with DeepSeek V4.1 Flash 1017 at the top on 97.6 for $0.0027 per solve. The older V4.1 Flash snapshot, which we could only reach through a third-party host, scored 63.9 and never delivered two of the real jobs. Qwen 3.8 Max 0902 and Tencent's Hy-3 tied at 92.1, and Hy-3 was far cheaper per solve, $0.0052 against $0.029. It was also the slowest model we ran, at a median 141 seconds per hard test, and its successor Hy4 was barely quicker at 126 seconds before running out of room on the editorial job.

Kimi K3 scored 88.1 at $0.092 per solve, the third highest cost in the results, and GLM-5.3 reached 87.3 despite scoring nothing on the editorial job. MiMo V2.5 Pro had the biggest gap between its hard tests and its real work, solving 26 of the 27 hard tests and then scoring 4 of 12 on the real jobs. Its audit figures were wrong, its SQL explanation was wrong and its article misstated a retry cost, while MiniMax-M3 scored 69.8 after producing nothing usable on the SQL and editorial jobs.

Everyone else

Nemotron 3.5 Lightning is cheap for a reason, and it finished on 7.1 after solving none of the 27 hard test versions. It also got the audit figures wrong, shipped a payment fix that still refunded the wrong charge and endorsed a misleading headline in its handover. Mistral Medium 3.5 went the other way on price as the most expensive configuration per solve, $0.179 for a score of 78.6. Nemotron 3 Ultra reached 73.8 with an SQL explanation that called valid SQLite syntax illegal, and Inkling scored 81.7 after losing all its editorial points to em dashes and an unsupported claim.

Answers that stood out

The most worrying answer came from Mistral Medium 3.5 in a harder version of the changing-an-answer test, where it gave 468 cents with a confidence of 100. It then accepted an editor's wrong correction and changed its answer to 644, again with a confidence of 100. Kimi K3's third attempt at the pagination repair still broke the brief's rule for empty pages after $0.42 across three tries, while GLM-5.3 Flash got the same repair right but needed 337 seconds and 29,905 tokens to do it, for $0.015.

DeepSeek V4.1 Flash 1017 gave us two of the strangest results in the round, starting with a harder version of the small fix on which it spent all 32,768 output tokens reasoning and never wrote an answer. With its thinking turned off, it later repaired the tool test's code correctly and then reported five tool calls when it had made six, at a confidence of 96.

What else we measured

Effort

We also ran six models at each effort setting they offer on 15 hard test versions, and more thinking mostly bought a bigger bill. GPT-6.1 Sol solved all 15 at Low for $0.008 per solve, while Claude Opus 5.5 at Max solved one more than at Low, 15 against 14, for 12 times the cost per solve. Sonnet 5.5 at Max solved fewer than at xhigh, 12 against 15, at nearly ten times the cost, and Qwen 3.8 Max solved its most at Medium and only 4 at xhigh. Gemini 4 Argon at High solved the same 15 as at Medium for 2.7 times as much.

ModelSettingSolved (of 15)Cost per solve
GPT-6.1 SolLow15$0.0082
GPT-6.1 SolMedium15$0.0090
GPT-6.1 SolHigh15$0.012
GPT-6.1 Solxhigh15$0.018
GPT-6.1 SolMax15$0.023
Claude Opus 5.5Low14$0.027
Claude Opus 5.5Medium15$0.036
Claude Opus 5.5High14$0.035
Claude Opus 5.5xhigh15$0.048
Claude Opus 5.5Max15$0.327
Claude Sonnet 5.5Low13$0.015
Claude Sonnet 5.5Medium13$0.016
Claude Sonnet 5.5High13$0.019
Claude Sonnet 5.5xhigh15$0.021
Claude Sonnet 5.5Max12$0.204
Gemini 4 ArgonLow13$0.010
Gemini 4 ArgonMedium15$0.013
Gemini 4 ArgonHigh15$0.036
DeepSeek V4.1 Flash 1017Thinking off5$0.0015
DeepSeek V4.1 Flash 1017Low14$0.0019
DeepSeek V4.1 Flash 1017High14$0.0041
DeepSeek V4.1 Flash 1017Max15$0.0030
Qwen 3.8 Max 0902Minimal8$0.029
Qwen 3.8 Max 0902Low8$0.033
Qwen 3.8 Max 0902Medium10$0.025
Qwen 3.8 Max 0902High8$0.021
Qwen 3.8 Max 0902xhigh4$0.047

Nobody overthought the easy questions either, since none of the 43 configurations in the first round spent a median of more than 1,000 tokens on them.

Price

We also compared the six models with the highest list prices, Claude Fable 5, Fable 5.1, Fable 5.2, Opus 5, Opus 5.5 and GPT-6 Astra, with six cheap ones: GPT-6 Luna, GPT-5.6 Luna, DeepSeek V4.1 Flash 1017, GPT-6.1 Terra, GLM-5.3 Flash and MiMo V2.5 Pro. Across eight hard tests the expensive group paid 8 to 42 times as much per solve, even though we counted only its first attempts and gave the cheap group its repairs. For that money it never solved more than one extra test version out of six, and on the pagination repair the cheap group solved more.

Hard testCheap six solvedExpensive six solved first timeCheap cost per solveExpensive cost per solveTimes as expensive
1: time zones56$0.0074$0.10314.0
2: pagination53$0.0071$0.13619.3
3: a small fix56$0.0039$0.04010.2
4: scheduling55$0.0081$0.0688.4
5: changing an answer65$0.0015$0.06141.7
6: a planted instruction66$0.0006$0.01729.0
7: refund policy66$0.0017$0.04827.9
8: table edits66$0.0023$0.03816.3

Hosts

Where a model is served changed the bill and the wait far more than the answers, and Kimi K3 shows it clearly: it solved the same 23 of 24 hard test versions on Baseten and on Chutes, but Chutes took about five times as long per solve. The earlier DeepSeek V4.1 Flash snapshot cost 2.2 times as much per solve through Relace as through Baidu for the same 23 solves. The 1017 snapshot went the other way, because Sail Research's lower-precision FP4 route cost a third as much per solve as DeepInfra's FP8 route and answered three times faster, with 23 solves against 22.

Newer versions

Most version upgrades cut the cost per solve without losing solves, 13 of the 16 we could compare, and some of the cuts were large. GPT-6.1 Terra, for example, spent 77% less per solve than GPT-5.6 Terra, although very few upgrades used fewer tokens once we allowed for normal variation between runs.

Confidence

Each hard answer included the model's own confidence from 0 to 100, and the wrong answers were nearly as confident as the right ones. In the first round, 37 of the 40 wrong answers that gave a confidence rated themselves 80 or higher, on average only about three points below the confidence on correct answers, so a confident answer told us almost nothing about a correct one.

Wrong answers in the first roundCount
Rated 80 to 10037
Rated below 803
No confidence given3

The same models handled the opposite situation without a miss, and all 86 answers to our two unanswerable everyday questions said the information wasn't there.

How the score works

Grades mean slightly different things for each kind of job, because a hard test can be retried with feedback while an everyday question can't. On the real jobs we graded both the automatic checks and our own reading of the write-ups, explanations and handovers behind them.

GradeHard testsEveryday questionsReal jobs
Good, 3 pointsSolved on the first attempt10 of 10 rightPassed the checks, and our reading found nothing wrong
Pass, 2 pointsSolved after a repair8 or 9 rightUsable after a small fix: one repair, an answer cut off but complete, or formatting to tidy
Not Good, 1 pointUnsolved, but scored 5 out of 10 or more5 to 7 rightPassed the checks, but our reading found a false claim or a wrong explanation, or the files couldn't be read
Fail, 0 pointsUnder 5 out of 10, broke the format rules, or hit the 32,768-token limitUnder 5Wrong, or nothing usable within the limit

A hard test's points are the average of its three versions, and a wrong hard answer could be repaired up to twice, with feedback that named the failing field or case group but never the answer. When a provider or connection failed on both permitted attempts, we left that job out of the score instead of counting it as a fail. That happened to Grok 4.7 on one version of hard test 2 and to the older DeepSeek V4.1 Flash snapshot on the payment and editorial jobs.

Hard test 9 needed a ruling from us, because its brief allows one tool call per reply and several models asked for more than one at once, which accounts for 29 of the 45 failed versions of that test. We count that as a fail, since the brief states the rule plainly, most models followed it, and a pipeline built to take one call at a time breaks when a model sends several. Cost, time and the model's own confidence are reported next to the score rather than counted in it, and when two models tie, the one that spent less per solve is listed first.

How we use the score in our ranking

Our Best AI Models ranking turns the score into a ceiling, so a model can only reach the tiers its score allows:

TGK ScoreHighest tier allowed
90 or moreSSS
80 to 89.9A+
70 to 79.9B
60 to 69.9C
40 to 59.9D
Below 40E

Within that ceiling, the tier reflects the hardest work we'd hand the model, and the order inside a tier comes from our daily work with it, judged on reasoning, reliability, cost and speed, and writing. That's how GPT-6.1 Sol and Muse Ember can score 100 and still rank below Claude Opus 5.5 at 97.6: the test shows all three can handle these jobs, and our own use tells us which one we trust with a messy project.

Limits of this test

Each task ran once per model, so a difference of a few points falls within the noise, and with six configurations on 100 the test can't separate the leaders until we add harder problems at the top. The tasks are also short, self-contained and built on invented data, which leaves out long sessions, large codebases and how a model's writing holds up over weeks of use, all of which our ranking judges from daily work instead. Most models ran at Medium reasoning, or at the provider's default where no Medium setting exists, so a model at its highest setting could score differently, and because each went through a single provider, the same model served elsewhere may behave differently.

The real jobs first ran with an 8,192-token output limit, and any job that ran out of room was rerun at 32,768 tokens, with the rerun replacing the original. The editorial job for Claude Opus 5, Opus 5.5, Fable 5.1 and Fable 5.2 ran in the Claude app instead of the API. The fresh-chat packet for those runs left out the five source notes, so all four models said so in their handovers and checked figures against the source map alone.

GPT-6.2 Sol and GPT-6.2 Astra ran their real jobs at 32,768 tokens from the start, and their longest reply used 2,344, so the higher limit made no difference to their results.

How we ran it

The API work went through OpenRouter between 12 and 17 November 2026, with each model pinned to a single provider, automatic fallback turned off and the reasoning setting recorded for each request. Hard tests and everyday questions could use up to 32,768 output tokens, and code ran in a sandbox against tests frozen before any model saw a task. Each request was billed to the model that made it, failures and retries included, with the charge checked against its receipt. Our general rules for tests like this, from fixing the task before the first run to keeping first results and failures visible, are in How to Compare AI Models, and the rest of our methods are on How We Know.

TGK privacy settings

Choose What TGK May Use

Optional tracking starts only when you allow it. You can change your choice at any time from Privacy settings at the foot of every page.

Essential Functions

Always available

Needed to deliver the site, keep it secure and remember this choice. Tools you use, such as saved preferences, may also store data in your browser.

Google Analytics counts visits and shows which articles are read and for how long. If you allow it, it sets cookies that can recognise your browser on later visits.

Google Ads measures visits that come from TGK’s own advertising campaigns. It does not turn on personalised advertising on TGK.

We keep your choice on this browser for up to six months. Google may process data outside the EEA. Read how your data is used.