On-Prem GPUs vs Cloud: Owning a B200 Server Beats Renting Above 37% Utilization

Owning an 8-GPU B200 server costs about $173,800 a year on published prices, which beats renting the same hours on demand once utilization passes roughly 37%. That is the on-prem GPU vs cloud break-even for a steady inference workload, with a four-year depreciation life. Below it, rent. Above it, buy.

The model: one formula and five published inputs

Enterprises running steady inference are weighing on-premises GPU servers against cloud capacity. Most of the comparisons we found online start from a vendor's total cost of ownership calculator. This one starts from rate cards and lists that anyone can open.

The question it answers is narrow. For one 8-GPU server running a flat, continuous workload, at what share of hours does owning cost the same as renting? We call that share the break-even utilization.

The five inputs

The model needs five numbers. Each one below comes from a page we opened on 13 October 2026.

• Hardware price. A reseller's starting price for an 8-GPU HGX B200 system: $487,737.
• Depreciation life. A judgement, not a published number. We use four years and test three and five.
• Power draw. NVIDIA's rated maximum for a DGX B200: 14.3 kW.
• Facility cost. CBRE's average colocation asking rate: $195.94 per kW per month, plus electricity at the US commercial average of 14.53 cents per kWh.
• Cloud rate. Per-GPU-hour on-demand prices from published rate cards.

The formula

Annual cost of owning is the sum of three terms. Capital divided by life, power draw times the colocation rate times 12, and power draw times 8,760 hours times the electricity price. Call that total O.

Annual cost of renting at full time is the cloud rate per GPU-hour times 8 GPUs times 8,760 hours. Call that R. Because on-demand cloud bills only for hours used, renting costs R times your utilization.

Break-even utilization equals O divided by R. Owning wins when your real utilization is higher than that ratio. Everything else in this post is an input to that division or a reason it might mislead you.

What owning costs for a year

The three owned-cost terms are hardware, space and power capacity, and electricity. They behave differently, so we price them one at a time.

The hardware line

NVIDIA does not publish list prices for its data centre systems. Its DGX B200 page links to "Get DGX" and shows no price. The nearest public number is a reseller's. Thinkmate lists three 8-GPU HGX B200 systems from $487,737 to $519,457, with H200 systems from $392,703 and B300 systems from $630,404.

We use the lowest B200 figure, $487,737. It is a starting price for a base configuration, so a production build with fast networking and storage will cost more. That makes our hardware line a floor.

Spread over four years, the hardware costs $121,934 a year. This is straight-line, with no resale value and no cost of capital. Both choices flatter the cloud, and we flag them again in the limitations section.

The facility line

A server that draws 14.3 kW needs a place to run. CBRE reported in February 2026 that the average asking rate for 250 to 500 kW requirements was $195.94 per kW per month, the fourth annual increase in a row. Primary-market vacancy was 1.4%.

At that rate, 14.3 kW of reserved capacity costs $2,802 a month, or $33,623 a year. A 250 kW block holds about 17 of these servers, so the model assumes you operate at cluster scale rather than renting one rack.

Electricity is the third term. The US Energy Information Administration put the average commercial price at 14.53 cents per kWh in July 2026. Running at rated maximum for 8,760 hours uses 125,268 kWh, which costs $18,201.

That electricity line is a ceiling. Servers rarely sit at rated maximum around the clock. We also cannot tell from CBRE's text whether its asking rate bundles metered power. So we show the model both ways.

Excluded: networking and storage fabric, staff, spares, cost of capital, resale value, taxes and any minimum-term charges in a colocation contract. Each of these raises the owned cost. The total is therefore a lower bound on everything except electricity.

What renting the same hours costs

Cloud GPU pricing is not one market. Three kinds of seller publish three kinds of rate, and the spread between them is wide enough to decide the answer.

GPU clouds with published B200 rates

Lambda lists B200 SXM6 instances at $6.69 per GPU-hour on an 8-GPU instance. That is $53.52 an hour for the server, or $468,835 across a full year.

CoreWeave prices by instance rather than by GPU. Its HGX B200 is $68.80 an hour for 8 GPUs, which works out to $8.60 per GPU-hour and $602,688 a year at full time.

The same silicon, two prices, a 29% gap. For an older GPU the spread is similar. Lambda's 8-GPU H100 rate is $3.99 per GPU-hour, CoreWeave's is $6.16, and the AWS figure below is $6.88. We would treat the choice of cloud as an input to the model, not a footnote.

Hyperscalers and reserved commitments

AWS does not render its per-instance prices in a page we could read, so we used Vantage, which mirrors the AWS price list. It showed $55.04 an hour for a p5.48xlarge, which is 8 H100 GPUs at $6.88 each, and $23.777 an hour on a three-year reservation. That is a 57% discount for committing to three years.

Rate cards also move. AWS cut P5 on-demand prices by 44% in June 2025. A model built on one quarter's prices has a shelf life, and we say so in the references.

CoreWeave advertises reserved discounts of up to 60% but publishes no rates. Lambda says to contact sales for reserved capacity. For B200 hardware, the committed price is therefore not public. We use "up to 60%" as a stated ceiling in the next section, not as a quote.

Break-even utilization, worked through

Now divide. Owned cost per GPU-hour at full time is $173,758 divided by 70,080 GPU-hours, which is $2.48. The cloud rate is what you pay per hour actually used.

If you use the server a fraction u of the year, your owned cost per useful GPU-hour is $2.48 divided by u. At 25% utilization that is $9.92. At 50% it is $4.96. At 90% it falls to $2.75.

The circles mark where the owned curve meets each cloud rate. Against Lambda's $6.69, owning wins above 37.1% utilization. Against CoreWeave's $8.60, it wins above 28.8%.

Those are the headline numbers, and they rest on two choices: the four-year life, and the electricity ceiling. Table 2 changes each.

Model output, not measured data. Spot is CoreWeave's $34.11 per hour for 8 GPUs divided by 8. Interruptible capacity is a poor fit for a steady inference service with a latency target, which is why the base case ignores it.

Why reserved capacity changes the question

Everything above compares ownership with on-demand cloud, which charges for hours used. Reserved capacity charges for every hour of the term. That moves the comparison from a utilization question to a plain annual-cost question.

Take the three-year AWS reservation. At $23.777 an hour it costs about $208,000 a year whether the eight H100s run flat out or sit idle. An owned server has the same shape: its cost is fixed too. Neither side gets cheaper per year when utilization falls.

So against a reservation, the owner wins or loses on the annual totals. Utilization still matters, but it hits both sides equally. A reserved GPU and an owned GPU at 20% use are both expensive per useful hour, and the ratio between them does not change.

Here is a concrete case using the only reserved ceiling a GPU cloud has published. If CoreWeave's full "up to 60%" applied to its B200 rate, a year would cost $602,688 times 0.4, or about $241,000. That is 39% above our owned total of $173,758. We would not bank on the full 60%, and the real quote is private.

Our view is that a reservation is the right comparison for any workload you can forecast a year out. On-demand is the right comparison only if you would genuinely shut the GPUs down when traffic drops. Few production inference teams do that.

How utilization behaves in real fleets

The model treats utilization as a dial you set. In practice it is an outcome, and the evidence on where it lands is not encouraging for owners.

What enterprises say they run at

VentureBeat Research surveyed 573 technical leaders at organisations with 100 or more employees in June 2026. It reported that about 83% of enterprises running their own GPUs said utilization was 50% or lower.

The figure was corrected from 86% after publication, so some secondary sites still repeat the older number. The article defines neither utilization nor the number of self-hosted respondents, and it asks readers to treat results as directional. We do too.

Read against our model, the pattern is uncomfortable. A 50% owner clears the 37% line against Lambda, and a 30% owner does not. If a large share of that 83% sits at 30% or lower, ownership is losing to on-demand rental in a lot of enterprises.

Average load is not peak load

The model assumes a flat workload. Real inference traffic has a daily shape and a weekly one. An owner has to buy for the peak, then pays for the trough.

That is the gap between average and provisioned capacity. If peak traffic is twice the overnight floor, a fleet sized for the peak will show low average utilization even when nothing is wasted. The honest number to plug into the formula is average load divided by purchased capacity, not the busiest hour.

It is also the strongest argument for a hybrid. Own enough capacity to cover the steady floor at high utilization, and rent burst capacity on demand. The floor is where the formula favors owning. The peaks are where it favors renting.

Where this model is weakest

Every post that argues one side of a build-or-rent decision should state where it fails. This one fails in five places, and some of them are large.

What the model leaves out

The hardware price is a reseller's starting price for a base configuration. NVIDIA publishes no list price, and the Thinkmate product titles do not name the GPU. We inferred B200 from the "HGX-B2" suffix and the site's filter counts, so treat that mapping as unverified until you see a quote.

The colocation rate is a wholesale asking rate for 250 to 500 kW. A single server is far below that block, and small deployments usually pay more per kW. CBRE also expects rates above $215 per kW per month for this band, a forecast from its midyear 2026 outlook that would raise the facility line by about 10%.

The power figure comes from NVIDIA's DGX B200 page. A reseller's HGX server uses the same eight-GPU baseboard but a different chassis, so its real draw will differ somewhat.

Cost of capital is missing, and it matters. Spending $487,737 up front ties up cash that would otherwise earn a return. Resale value cuts the other way: a four-year-old accelerator is not worth zero. Staff is missing too. Someone has to rack, patch, monitor and replace parts, and no published rate card prices that. How long a GPU stays economically useful is itself disputed, and the argument is laid out in our look at how GPU depreciation schedules shape reported earnings.

The biggest omission is output. A GPU-hour is not a token. A B200 does more work per hour than an H100, and a rate card tells you nothing about throughput on your model. Cost per useful GPU-hour is a proxy, and the real unit is cost per million tokens served.

Finally, the cloud side hides discounts. Enterprise agreements, committed spend programs and marketing credits all lower the list rate, and none of them are published. A buyer with a negotiated 30% discount has a different break-even than this post shows. The break-even moves up by the same proportion.

What this means for your decision

Three conclusions survive the caveats. We hold them with moderate confidence, and we would change them if a quote contradicted the inputs.

First, steady, forecastable inference at high utilization favors owning. At 70% utilization the owned cost is $3.54 per used GPU-hour in our base case, about half the Lambda rate. That is a large margin even if the real hardware bill is 30% above our floor.

Second, lumpy or uncertain demand favors renting. If you cannot say whether you will use 25% or 60% of a server next year, the downside of owning is asymmetric. A server at 25% costs you $9.92 per used hour, 48% more than Lambda's rate, and you own it for four years.

Third, run the hybrid before you choose a side. Buy for the floor, rent for the peaks, and revisit each year as rate cards fall. The June 2025 AWS cut is the reminder that the cloud side of this formula is moving down while the hardware side is not.

The wider context is a capital cycle in which suppliers of both options are financing growth with debt. We cover the cloud-provider side of that in the analysis of CoreWeave's debt stack, and the spending behind it in the payback math on hyperscaler capex. If you buy tokens through an API instead of GPU hours, the unit economics change again, as our piece on inference costs and AI margins explains.

One more option deserves a line. Owning hardware only pays if the model you run is one you are free to run. Weights, licences and support differ between providers, which is the subject of our comparison of open-weight models in production.

Frequently asked questions

Where to start this week

Copy the table above into a spreadsheet and replace each input with your own quote. Start with the two that move the answer most: the hardware price and the depreciation life your finance team will accept. Then compute O divided by R against your actual cloud bill rate, not the list rate.

Next, measure one real number. Take 30 days of GPU metrics from your busiest cluster and compute average load divided by purchased capacity. If the result is under 40%, do not buy more hardware until you know why.

Related reading