Small Language Models Priced Per Task: Phi-4, Gemma 3 and Mistral Small

Running a frontier model on a narrow task costs 30 to 37 times more than running Phi-4 on the same task, at October 2026 published rates. That gap is real and it is the whole pitch for small language models. The more interesting finding is the row underneath it: the cheapest closed model now sits within 2 times of the open small models, which changes what the decision is actually about.

If you only read one paragraph, read this. For a fixed-label classification or fixed-schema extraction job, a small language model at a marketplace host costs roughly 3 to 4 cents per 1,000 short calls, and a frontier model costs about a dollar. Before you migrate, measure accuracy on 500 of your own items, because every accuracy number in public is either vendor-reported or measured on someone else's data.

The published price sheet for small language models

A small language model, for this post, is an instruction-tuned model under roughly 30 billion parameters that a single high-end GPU can serve. NVIDIA researchers draw the line lower, describing most models below 10 billion parameters as small language models. We stretch it to 27 billion because that is where Gemma 3 and Mistral Small sit, and those are the models teams actually shortlist.

We priced three small models and two closed models from Anthropic. Every rate below was read from the vendor or host's own pricing page on 9 October 2026. None comes from an aggregator.

Sources: DeepInfra pricing page; Microsoft Azure Retail Prices API meters "Phi-4-Input Tokens" and "Phi-4-Output Tokens" at $0.000125 and $0.0005 per 1,000 tokens; Mistral API pricing docs; Anthropic API pricing page. Gemma has no paid first-party rate: Google's Gemini API pricing page lists Gemma 4 on the free tier only, with the paid tier marked not available.

The first-party rates are 1.8 to 3.6 times the marketplace rates for the same weights. Phi-4 output costs $0.50 per million tokens on Azure and $0.14 on DeepInfra. That spread is the price of a direct contract, enterprise terms and a data processing agreement with the model's own publisher.

My view is that most teams should benchmark on the marketplace price and buy on the first-party price. The cheaper number tells you what the weights cost to serve. The dearer one is usually what your procurement team will sign.

Mistral's own docs list Mistral Small 3.2 as deprecated on 30 April 2026 and retired on 31 July 2026 from its API. DeepInfra still serves it. This is the quiet advantage of open weights: a hosted closed model leaves when its vendor says so, and an open model stays available for as long as someone is willing to run it. The terms behind that freedom are covered in the clause-by-clause read of open-weight model licences.

The small language model test, task by task

Three tasks make up most narrow production LLM traffic. They are classification, extraction and summarisation. Each has a fixed prompt, a fixed scoring rule and a fixed token budget. The prompts below are the exact strings. Swap in your own data at the placeholder and change nothing else.

Run every model at temperature 0, with the output cap shown, on the same 500 items per task. Use 500 because it is enough to see a 3-point accuracy difference with some confidence. It is also small enough to label in two days.

Task 1: ticket classification, five labels

The budget is about 450 input tokens and 10 output tokens per call. Score exact match against your human label, and report macro-F1 alongside accuracy so a model that ignores the rare class cannot hide behind the common one.

You are a support ticket router. Classify the ticket into exactly one
label from: billing, bug, feature_request, account_access, other.
Reply with the label only, lowercase, no punctuation.

Ticket:
{ticket_text}

Task 2: invoice field extraction to JSON

The budget is about 1,200 input tokens and 150 output tokens. Score field-level exact match on the five fields, and separately report the share of responses that parse as valid JSON. A model that is right on the fields 95% of the time and returns broken JSON 4% of the time needs a retry path, and that retry has a price.

Extract these fields from the invoice text. Return JSON only, matching:
{"vendor_name": string, "invoice_number": string,
 "invoice_date": "YYYY-MM-DD", "total_amount": number,
 "currency": "ISO 4217 code"}
If a field is absent, use null. Do not infer values not in the text.

Invoice text:
{invoice_text}

Task 3: call transcript summary for an account manager

The budget is about 3,000 input tokens and 250 output tokens. Score two things. First, format compliance: exactly 3 bullets, each under 25 words. Second, faithfulness: every claim in the summary appears in the transcript, checked by a frontier model acting as judge and then spot-checked by a person on 50 items.

Summarise the call transcript below for an account manager in exactly
3 bullet points of at most 25 words each. Cover: the customer's main
problem, what was agreed, and the next action with its owner.
Use only facts stated in the transcript.

Transcript:
{transcript}

If you want to build the scoring side properly rather than as a spreadsheet, the structure is set out in the guide to building an LLM eval suite from your own tickets. Fixing the prompts turns the cost side of this test into pure arithmetic. Only quality is left to measure.

Small model vs frontier model: cost per 1,000 tasks

Cost per 1,000 tasks is input tokens times input price plus output tokens times output price, divided by 1,000. Using the token budgets above and the rates in the price sheet, here is what each model charges for each task.

Token budgets are test design parameters chosen by Zan Digital, not measurements. Your prompts will differ. Tokenisers also differ across model families, so the same text produces different token counts on each model. Treat the table as a ratio, not an invoice.

At 300,000 extraction tasks a month, about 10,000 a day, Sonnet 5.5 costs $1,170 and Phi-4 on DeepInfra costs about $32. The saving is roughly $1,140 a month. That is real money, and it is also smaller than the cost of maintaining a second provider integration, a second eval suite and a second on-call path.

At 10 million tasks a month the same arithmetic gives $39,000 against $1,050. That is where the migration pays for itself without argument. I would not move a workload under a million calls a month to a small model for cost alone; I would move it for latency, residency or control, and take the saving as a bonus.

Anthropic's batch API halves both input and output prices. Run the extraction job as a batch on Sonnet 5.5 and it falls to $1.95 per 1,000 tasks, still 18.6 times Phi-4 on DeepInfra. Mistral and Google also advertise 50% batch discounts. Prompt caching cuts the repeated instruction portion further on every vendor that offers it, which matters most for classification, where the instructions are a large share of each call. Falling token prices can still produce rising total bills. That separate trap is set out in the analysis of why cheap tokens keep producing bigger AI bills.

What the small language model accuracy evidence says

This is where honesty matters most. We publish the harness and the cost arithmetic. The accuracy evidence below is published research and vendor model cards. None of it was measured on your data or on ours.

The fine-tuning result that started the argument

The largest public test of fine-tuned small models is Predibase's LoRA Land, published in April 2024. The team fine-tuned 10 base models, all between 2 and 8 billion parameters, on 31 tasks, producing 310 models. 224 of the 310 fine-tuned models surpassed GPT-4, and 301 surpassed their own base model.

Taking the best fine-tuned model per task, the average score was 0.756 against 0.661 for GPT-4. GPT-4 still won 6 of the 31 tasks, including code generation, MMLU and text-to-SQL. Read that list carefully. Every task GPT-4 kept needs broad knowledge or multi-step construction.

Model card scores, and what they cannot tell you

Microsoft's Phi-4 technical report gives the 14-billion-parameter model 70.4 on MMLU-Pro against 73.0 for GPT-4o. Google's Gemma 3 report gives the 27B instruction model 67.5 on the same benchmark. Mistral describes Mistral Small 3 as scoring over 81% on MMLU.

These scores are self-reported by the model's own developer. They also measure general knowledge and reasoning, which is the opposite of what you will ask a small model to do in production. The same Phi-4 table shows a score of 3.0 on SimpleQA, a factual recall test, against 39.4 for GPT-4o. A small model knows less. On a classification task where the answer is in the input, that rarely matters. On a summary where the model fills a gap from memory, it matters a great deal.

The judgement worth carrying away

I think the public evidence supports one claim strongly and nothing more. On a narrow task with the answer present in the input, a fine-tuned small model usually matches or beats a general frontier model. It does not support the claim that an untuned small model matches a frontier model on your task. That second claim is the one most vendor decks make. Model routing can hide a quality gap behind an average. That gap is examined in the measurement of what routing to cheaper models actually saves.

Small language model latency and context limits

Latency is the argument for small models that survives the Haiku price cut. NVIDIA's researchers state that serving a 7 billion parameter model is 10 to 30 times cheaper in latency, energy and FLOPs than a 70 to 175 billion parameter model. Mistral claims 150 tokens per second for Mistral Small 3 and says it runs more than 3 times faster than Llama 3.3 70B on the same hardware. Both are developer claims.

Marketplace latency also depends on the host's load, region and queue, which no model card controls. That is why the harness records time to first token and total time per call, at the 50th and 95th percentile, from your own region. A p95 is the number your users feel.

Phi-4 was trained with a 16,000-token context. Gemma 3 27B supports 128,000 tokens, and DeepInfra lists Mistral Small 3.2 at 125,000. For the 3,000-token summary in this test, all three fit. For a 40-page contract, Phi-4 does not, and no price advantage rescues a model that cannot read the document.

Every price in this post is per token on a shared API. Run the weights on your own GPU and the cost becomes a fixed hourly rate divided by throughput, which favours high, steady volume and punishes idle time. Mistral states that a quantised Mistral Small 3 runs on a single RTX 4090. Whether that is cheaper than $0.075 per million input tokens depends entirely on utilisation, and the honest way to answer it is to run the harness on both and divide.

Where this test is weakest

Four weaknesses, stated plainly, because a reader deciding where to send production traffic deserves them before the decision rule.

We publish the method, not a leaderboard. This post does not report accuracy from our own run on the three tasks. We do not hold a labelled production dataset we can publish, and a leaderboard on synthetic tickets would tell you about synthetic tickets. The cost figures are exact arithmetic on published rates. The accuracy figures are other people's measurements on other people's data.

The token budgets are assumptions. Real prompts carry system instructions, few-shot examples and retrieved context. A classification prompt with 10 worked examples could run to 2,000 input tokens, which changes every number in the cost table, though not the ratio between rows much.

Marketplace prices move and hosts differ. DeepInfra's rate is one host's list price on one day. Hosts can serve the same weights at different numerical precision, which can shift quality. Check the precision a host states for the endpoint you call before you compare its output to a first-party API.

The best fine-tuning evidence is aging. LoRA Land's baseline was a 2023 GPT-4 release. Frontier models have improved since, and so have small ones, and no public study of equal size has repeated the comparison at today's frontier. The strongest case against this post is that the current frontier would close more of those 31 tasks.

A decision rule for small model vs frontier model, by task type

Here is my rule. It assumes you have run the harness on 500 of your own items and have accuracy numbers to put in it.

Classification and extraction: start small, escalate on failure

If the open small model lands within 2 accuracy points of the frontier model on your 500 items, ship the small model. Route any response that fails validation, an unknown label or unparseable JSON, to the frontier model as a retry. On extraction, that retry path is where most of the remaining errors get caught, and it costs a few cents per thousand.

Summarisation: let context length and audience decide

For internal summaries under 16,000 tokens, any of the three works if faithfulness holds on your 50-item spot check. For long documents, Phi-4 is out on context alone. For anything a customer reads, I would start on the closed small tier or frontier until faithfulness on your own data is proven. One invented commitment in a customer summary costs more than a year of token savings.

Reasoning and open-ended work: stay on the frontier

Multi-step reasoning, broad-knowledge questions and agent planning are where LoRA Land's GPT-4 wins clustered and where the SimpleQA gap bites. Keep these on a frontier model, and revisit only when a small model passes your harness. The wider argument about where agent steps can shift to cheaper models is set out in the case that multi-agent systems are overrated.

Frequently asked questions

How to run it this week

Pull 500 real items for the one task that sends you the most LLM calls. Label them, paste the matching prompt from this post, and run Phi-4, Gemma 3 27B and Mistral Small 3.2 on a marketplace host alongside the model you use today. A day of work and well under a dollar of tokens gets you the only accuracy number that matters, which is yours.

Then put that number in the decision rule. If the small model lands within 2 points, price the migration at your real monthly volume before you schedule it, because at low volume the saving may not cover the second integration.

Related reading