Inference Provider Comparison: Same Open Model, Same Price, Different Bill

Three of the four best-known inference providers charge exactly the same list price for the same open model. Together AI, Fireworks AI and Groq all list gpt-oss-120b at $0.15 per million input tokens and $0.60 per million output tokens. An inference provider comparison that stops at the rate card therefore tells you almost nothing. The bill is decided by caching discounts, batch terms, throughput, rate limits and what happens to your prompts after the request ends.

This post is written for the practitioner: the engineer or technical lead who has to pick an endpoint for an open model this quarter and defend the choice. The direct answer: for gpt-oss-120b, Baseten has the lowest list price and Fireworks has the cheapest cached input and a published batch discount. Groq is the fastest by a wide margin. Together AI is the only one of the four that retains prompts unless you change a setting. Pick on workload shape, not on the headline token price.

The inference provider rate card: one model, four list prices

The table below is the reference this page exists to maintain. Every figure was read from the provider's own pricing page, model page or documentation on 16 October 2026. Prices are per million tokens for serverless, pay-as-you-go access with no contract.

Fireworks also sells a Priority tier for the same model at $0.18 input, $0.018 cached and $0.72 output. Groq's batch discount does not stack with its caching discount. Together's batch documentation lists only one model at 50% off and states that models not listed run at standard rates.

The first thing the table shows is convergence. Three companies with different hardware, different funding and different go-to-market strategies landed on an identical number. That is not a coincidence. It is the price the market has settled on for a model that anyone can download and serve.

The second thing it shows is that Baseten undercuts that anchor by a third on input and a sixth on output. My read is that this is a customer acquisition price for its Model APIs, which sit in front of a business mostly built on dedicated deployments. A buyer should take the discount and should not assume it is permanent.

A list price is the ceiling of what you pay per token, not the average. Four columns move the real number more than the headline does: the cached input rate, the batch rate, the tokens you can actually push through per minute, and the speed at which those tokens come back. The rest of this post works through each one.

The rate card also leaves out cheaper hosts. Artificial Analysis's own blended price ranking for this model, weighted for agentic use with cache hits, puts CoreWeave at $0.04 and DeepInfra at $0.05 per million tokens. None of the four providers here appears in that top five. Brand recognition in this market is not the same thing as price leadership.

Why gpt-oss-120b is the fair test model

A like-for-like comparison needs one model that every provider serves under the same name. That is harder to find than it sounds. Together AI's serverless catalogue does not list Kimi K2 or DeepSeek V3, Fireworks does not name Llama 3.3 70B on its pricing page, and Baseten lists no Llama or Qwen model at all.

gpt-oss-120b is the exception. OpenAI released it with open weights, it ships in a native 4-bit format, and all four providers expose it under the same slug. Fireworks describes it as having 117 billion total parameters with 5.1 billion active per token. That small active footprint is why it is cheap to serve, and why its price has become a market benchmark.

Same weights do not guarantee the same model

Providers can serve the same checkpoint with different quantisation, different speculative decoding and different default reasoning settings. Together AI lists its deployment as MXFP4, which matches the native release format. The others do not all publish the detail. Treat "same model" as a strong assumption, not a proven one, and run your own evaluation before you switch production traffic. The method for that is set out in the analysis of the quality gap that model routing hides.

Caching and batch move the bill more than list price

Many workloads repeat a long system prompt, a tool schema or a retrieved document across calls. In that case a large share of your input tokens can be served from cache. For agent loops this share is often the majority. The cached rate then matters more than the list rate.

The spread here is wide. Fireworks charges $0.015 per million cached input tokens, a 90% reduction. Groq charges $0.075, a 50% reduction, and cached tokens do not count against Groq's rate limits. Together AI's prompt caching documentation states that models without a cached price bill every input token at the standard rate, and gpt-oss-120b has none listed. Baseten's pricing table also shows no cached rate for this model.

A worked monthly bill

To make the differences concrete, take one workload: 100 million input tokens and 20 million output tokens a month. That is a modest production agent. The table prices it three ways. The 60% cache hit rate in the second column is my assumption for illustration, not a measured figure.

Arithmetic on published list rates only. Excludes enterprise discounts, free credits and taxes. Baseten and Together columns stay flat because neither publishes a cached or batch rate for this model. The cache hit rate is illustrative.

Batch is the largest single discount on the table

Groq's batch documentation states a 50% discount against synchronous pricing with a completion window you set between 24 hours and 7 days. Fireworks bills batch inference at 50% of serverless pricing on both input and output. For overnight classification, evaluation runs and backfills, that halves the bill on either provider.

Together AI's batch API exists and has generous capacity, with up to 50,000 requests per batch and 30 billion tokens enqueued per model. But its documentation lists a single discounted model and says everything else runs at standard rates. If batch is a large share of your volume, that one line in the docs costs more than any list price difference in this comparison.

Throughput: the 4.5x gap that list prices hide

Price per token says nothing about how fast those tokens arrive. For an interactive product, speed is the user experience. For a reasoning model, it also sets latency, because the model generates hidden reasoning tokens before it writes the answer.

Artificial Analysis measures this independently. Over the 72 hours before 16 October 2026, its median output speed for gpt-oss-120b was 474 tokens per second on Groq, 276 on Baseten and 105 on Together AI. Fireworks is not among the 17 providers it tracks for this model, so there is no independent figure for it here. Groq's own model page claims about 500 tokens per second, which the independent measurement broadly supports.

Why speed compounds on a reasoning model

gpt-oss-120b thinks before it answers. Artificial Analysis reports total time to return a 500 token answer, including reasoning, of about 6 seconds on Groq, 9 seconds on Baseten and 24 seconds on Together AI. Baseten had the fastest first chunk of the three, at 0.20 seconds.

A 24 second wait and a 6 second wait are different products. Neither number appears on any rate card. If your use case is a chat interface or an agent with a human waiting, I would treat throughput as the primary selection criterion and price as the tiebreaker.

Rate limits decide whether the price is available to you

A low price you cannot buy in volume is not a low price. The four providers take very different approaches to publishing limits, and the differences are larger than the price differences. The wider case for treating limits as part of the price is made in the 5-test standard for rate limit transparency.

RPM is requests per minute, TPM tokens per minute. Groq's entry figures come from its rate limits page and its Developer figures from its models table. Fireworks limits grow adaptively by up to 25% at a time toward the tier ceiling.

Together publishes the least

Together AI's rate limit documentation contains no request or token figures. It says serverless performance is best effort and that under high demand you may receive a 429 or 503 response. For guarantees it points to provisioned throughput, which is sales-only, priced at a flat $0.05 per PTU per minute on a minimum one month term, and which does not currently list gpt-oss-120b.

I do not think this is necessarily worse service. It may simply be honest about how shared serverless capacity behaves. But it means you cannot plan capacity from the documentation, and that cost lands on your engineering team as retry logic and fallback routing.

Baseten's limits are the tightest at the paid tier

Baseten has the lowest token price and the lowest published ceiling. Its Pro tier allows 120 requests and 1 million tokens per minute. Fireworks at Tier 3 allows 6,000 requests per minute account-wide. For a batch-heavy or high-concurrency workload, the cheapest per token rate may simply not be reachable without an enterprise contract.

Data retention defaults across the four providers

Retention is where these providers differ most, and where the difference is binary rather than a matter of cents. Three of the four state they do not keep inference content by default. One does. The provider-by-provider view of the closed-model labs is in the ranking of AI vendor data retention terms. This section covers the open-model hosts that ranking does not.

Together AI: on by default, off with one setting

Together AI's documentation is explicit. Unless your organisation enables zero data retention, Together stores the prompts you send and the responses models return, and may use them for product improvements. An organisation admin can switch storage off under privacy settings. That change is not retroactive, so anything sent before it stays under the old terms.

Training is a separate control and is off by default. The fine print worth reading is the passthrough setting. Passthrough models forward requests to third-party providers under their own policies, and that setting is on by default. Turning storage off also disables passthrough models.

Groq, Fireworks and Baseten: off by default, with exceptions

Groq states that it does not retain customer data for inference requests by default. It logs inputs and outputs only to troubleshoot errors or investigate suspected abuse, and keeps those logs for up to 30 days. Any customer can enable zero data retention, which removes that exception and disables batch.

Fireworks states that it has zero data retention by default and does not log prompt or generation data for open models without explicit opt-in. Its Responses API is the exception, storing conversations for 30 days unless you set store to false. Baseten's documentation says it does not store model inputs or outputs by default for synchronous inference.

Where this comparison is weakest

This page compares one model on four providers at one point in time. Each of those constraints limits what it can tell you, and I would rather name them than have you discover them in production.

Prices are a snapshot. Every figure was read on 16 October 2026. Open model prices fall quickly and providers rarely announce changes. That is why this table is scheduled for quarterly re-verification rather than treated as settled. The broader pattern of falling unit prices and rising total bills is covered in the analysis of why cheap tokens still produce rising AI bills.

Fireworks has no independent speed figure. Artificial Analysis does not track it for this model. I have not substituted Fireworks' own marketing numbers, so the throughput section is genuinely incomplete for one provider in four.

Quality is assumed equal, not measured. The same weights can produce different outputs after different serving optimisations. I have not run an evaluation across the four endpoints, and nothing in this post shows that they score identically on your task.

The documentation is not always internally consistent. Groq's rate limits page and its models table give different figures that appear to describe different plans, and the page labelling does not make clear which is which. Baseten's zero retention statement refers to synchronous inference without naming its Model APIs product. In both cases I have reported the most likely reading and flagged it.

List prices are not what large buyers pay. Every provider here sells enterprise contracts and committed throughput. Above a certain volume the negotiated rate replaces the rate card, and this comparison stops being the relevant one.

Which inference provider fits which workload

There is no single winner, and any comparison that crowns one is selling something. The useful output is a mapping from workload shape to provider. Here is mine, based only on the published terms above.

Interactive and latency-sensitive: Groq. It is roughly 4.5 times faster than Together AI on the same model at the same list price. For a product where a person waits on a reasoning model, that difference is the product.

Agent loops with long repeated context: Fireworks. The $0.015 cached rate is the lowest here by a factor of 5. A workload that resends the same tool schema on every turn will see that in the bill. The 6,000 request per minute ceiling also leaves room to scale.

Low-volume, cache-light, cost-first: Baseten. It has the lowest list price, a fast first token and no default retention. The Pro tier cap of 120 requests per minute is the constraint to plan around.

Together AI fits teams already on its platform for fine-tuning or dedicated GPUs, where one vendor relationship is worth more than a few dollars a month. It is the hardest of the four to justify on serverless gpt-oss-120b alone. Before you consolidate on any of them, read the breakdown of what switching model providers actually costs. Because the model is open, switching hosts is cheaper than switching labs, which is the real advantage of this whole category.

When to stop paying per token

Serverless pricing stops being the right comparison at sustained high volume. Together AI lists dedicated H100 capacity at $5.49 per GPU-hour, against roughly $6.50 an hour on Baseten. Whether that beats per-token pricing depends almost entirely on utilisation, and the method is set out in the GPU break-even utilisation model. The open model choice itself is covered in the comparison of open weights models in production.

Frequently asked questions

Where to start this week

Pull last month's token logs and calculate one number: what share of your input tokens repeat across calls. If it is above half, the cached rate is your real price and Fireworks or Groq should be on your shortlist regardless of what you pay today.

Then, if you are on Together AI, open your organisation's privacy settings and check whether storing prompts and responses is still switched on. It takes two minutes, and it is the only decision in this post that cannot be undone after the fact.

Related reading