The cheap model that costs dearly: how to verify it with your own data

Illustration generated with image2. Not a product screenshot.

Every time a model appears with a lower price per token, someone proposes migrating. The proposal usually rests on a spreadsheet with two columns and a subtraction. The problem is that the subtraction isn’t the one that appears on the invoice.

Kimi K3.1 Pro is advertised at 2 dollars per million input tokens and 10 per million output, versus 3 and 15 for the previous model, kimi-k3. A third cheaper on both sides. With that information, the decision seems obvious.

It isn’t. In seven tasks measured from the same account on the same day, the new model won five and lost two, and the two losses were in work that runs daily.

Where the difference comes from

The invoice doesn’t charge per published token, it charges per token generated. And these models bill internal reasoning as output tokens.

A model that thinks more before answering produces more billable tokens even if the visible answer has the same length. In the code repair task, the new model generated 1,545 output tokens, of which 1,468 were reasoning. The previous one generated 328. The visible answer was identical in both cases: 77 tokens.

There’s the mismatch. Five dollars of discount per million don’t compensate for fourteen hundred tokens of thinking.

What was measured

Seven tasks, two models, one day. Same statement for both, same documents, without adjusting the prompt per model. The goal was to compare the cost of migrating under equivalent conditions, not to get the best possible response from each one.

Input sizes ranged from 181 to 61,576 tokens. The total cost of the experiment, fourteen calls, was 1.14 dollars.

Task Input Output kimi-k3 Output K3.1 Pro Cost kimi-k3 Cost K3.1 Pro
Search in 55k document 61.567 279 240 $0.189 $0.126
Repair code 181 328 1.545 $0.0055 $0.0160
Tool call and calculation 540 197 101 $0.0046 $0.0021
Summarize 55k document 61.576 4.102 6.098 $0.246 $0.184
Search in 12k document 11.786 1.048 2.000 $0.0511 $0.0436
Summarize 12k document 11.773 2.530 8.000 $0.0733 $0.1035
Answer 12k email 11.789 882 2.258 $0.0486 $0.0462

Each row normalized to its maximum value. Measurement from 26-10-2026.

What each number means

The three data points aren’t worth the same and it’s best not to mix them.

Input is the only stable number: it depends on the document and the tokenizer, not the model. If you change models and the same text goes from 11,000 to 14,000 tokens, that difference is from the tokenizer and needs to be measured, not calculated.

Output is the number that decides. It includes the answer and the reasoning, and that’s why it almost never matches the length of the text you see on screen.

Reasoning is the number nobody looks at. In these tasks it represented between 45 and 95 percent of the output depending on the type of work, and it’s why two models with the same price can bill three times as much.

The rule that comes from this

The two price lists reduce to two expressions, with input and output in millions of tokens:

  • kimi-k3: 3 × input + 15 × output

  • K3.1 Pro: 2 × input + 10 × output

The new model is worthwhile when the second is less than the first. Subtracting, part of the input term cancels out:

10 × K3.1 Pro output − 15 × kimi-k3 output < input

Both sides are token counts, so they compare directly. If the left side is negative, the new model wins regardless of how much you send.

This inequality got the sign right in all seven tasks, including the two that came close to the limit: the 55k summary with −550 versus 61,576 input, and the 12k answer with 9,350 versus 11,789.

How to replicate it in an hour

You don’t need to set up anything special. Four steps and a document of your own.

Choose a long document from your work, not a sample article: length and density change the token count.

Define four types of task: search within that document, repair a code snippet, call a tool with calculation, and summarize the same document. Those four cover almost all real work.

Send the same statement to both models, without tweaking it for each one. What’s being compared is the cost under equivalent conditions.

Finally, read the usage block of the response, not the text: input, output, and how much of that output was reasoning. Multiply by published rates and subtract.

The cost at scale

The per-task numbers seem small until you multiply them.

With 200 long documents a day, 22 days a month, the previous model costs around 1,082 dollars and the new one around 810. The difference is 272 dollars a month, about 3,260 a year. For that volume, changing the workflow makes sense.

With 20 code repairs a day, the same 22 days, the previous one costs 7 dollars and the new one 2.4. The difference doesn’t reach 5 dollars a month. There the decision should be made by response quality, not price.

The conclusion is not that the new model is cheaper or more expensive. It’s that the question of price only makes sense after you know how many times the task runs.

Where the new model failed

Empty response with full invoice. With an output limit of 8,000 tokens, the summarization task consumed 7,997 in reasoning, returned empty content, ended with reason length, and charged 0.1035 dollars after 229.6 seconds. Raising the limit to 16,000 completed the same task in 150.8 seconds for 0.184 dollars.

What matters isn’t the failure, but that it doesn’t generate an error. The combination of empty content and reason length is the only signal; a dashboard that only watches status codes stays green.

The temperature parameter is rejected. The same request with temperature 0 or 0.6 returns HTTP 400, and the message doesn’t indicate which field is extra. Removing it restores the correct response.

Limits of this test

Four things are not verified here and it’s worth not taking them for granted:

  • The speed advantage of the highspeed variant.

  • Behavior with a million tokens of context actually in use.

  • How input images are tokenized.

  • The drop in hallucination from 23.4 to 8.3 percent, which is an internal figure without independent replication.

Also, the data comes from a single day and a single account. Reasoning token counts change with the prompt and with model updates, so the conclusion holds on the method, not on the exact numbers.

For discussion

What interests me about this thread isn’t whether the model is good, but how the work of each one is distributed.

If your load looks more like searching inside long documents, the rule says migrate: input dominates and the input price is a third lower. If it looks more like repairing code or summarizing medium texts, the rule says stay, and the savings disappear in the reasoning.

Has anyone measured the proportion of reasoning tokens to the total in your own prompts? In these tests it ranged from 45 to 95 percent depending on the task type, a spread much larger than what the presentation figures suggest.

And for anyone already using both models: did you divide the tasks or keep just one for simplicity? The arithmetic favors dividing; operations favor having fewer pieces.

Both models are available on the same endpoint; current prices and availability are listed in the model catalog.

Prices and specifications according to the October 26, 2026 announcement; token counts and costs measured that same day.