Cloud GPU Pricing November 2026: H100, H200, B200 and MI300X Per-Hour Costs Across Hyperscalers and Neoclouds
An NVIDIA H100 hour costs $12.29 on Azure’s flagship ND H100 v5 list price, $6.88 on AWS, and $4.39 on RunPod’s secure cloud. That is a 2.8x spread for the same silicon on the same day. Cloud GPU pricing in late 2026 is not one market; it is five overlapping ones, each with its own rules on commitment, interruption and what the hourly number actually buys.
It matters now because the long slide in rental prices has reversed. A public price index that tracks dozens of providers puts the median H100 on-demand rate up 31% year on year, AWS raised Capacity Block prices in January, and two neoclouds have published new rate cards that take effect within three weeks, one of them an explicit increase. Last year’s budget assumptions are probably wrong.
You will leave with a normalised per-GPU-hour table built from list prices fetched on 14 November 2026, the instance math behind every row, a methodology you can rerun, an illustrative cost-per-million-tokens model, and a decision guide.
What this covers: methodology, an H100 price per hour comparison, H200 and B200 cloud pricing, MI300X, commitment and spot tiers, neocloud GPU pricing, tokens-per-dollar, TCO beyond the GPU, and how to choose.
Context and Background
Three years ago the GPU cloud comparison was simple: hyperscalers had the capacity and the neoclouds had the price. In 2023 and early 2024, H100 capacity was rationed. Buyers signed multi-year contracts at whatever rate the provider asked, and on-demand H100 instances on the big three were often simply unavailable.
Then supply caught up. Through 2024 and into 2025, specialised GPU clouds such as CoreWeave, Lambda, Nebius, Crusoe and RunPod expanded quickly, and marketplace H100 rates fell sharply. AWS responded in June 2025 with the largest GPU cut in its history. Its official announcement cut P5 (H100) on-demand prices by 44% and P5en (H200) by 25%, effective 1 June 2025. That is why the AWS p5.48xlarge list price is $55.04 today, down from roughly $98 before the cut.
The 2026 story runs the other way. Blackwell (B200, B300, GB200) is ramping, but frontier training runs and inference demand for agentic workloads are absorbing every rack. The GetDeploying GPU price index reported for the week of 9 November 2026 that median rental prices across 22 GPU models rose 12% year on year, with 17 models up and 5 down. Its median H100 on-demand rate was $3.93 across 44 providers, with GPU clouds at $4.50 and hyperscalers at $7.89.
Providers have said the quiet part out loud. CoreWeave’s CFO said on the Q1 2026 earnings call that prices were “increasing across the board from Ampere to Hopper to Blackwell”, as reported in coverage of the call. AWS raised EC2 Capacity Block prices by about 15% on a Saturday in early January 2026, according to The Register. Crusoe’s pricing page now shows higher dedicated-endpoint rates from 1 December 2026.
This matters for anyone who modelled AI unit economics on 2025 numbers. Our analysis of hyperscaler capex and AI compute economics argued that the capex wave would eventually push rental prices down. That may still happen when Blackwell supply saturates, but it has not happened yet. The benchmark below is a snapshot of where list prices actually sit, not where forecasts said they would be.
Cloud GPU Pricing Benchmark: Methodology and Normalised Results
The short answer: in November 2026 an H100 lists at roughly $4.40 to $6.20 per GPU-hour on neoclouds and $6.88 to $12.29 on hyperscalers. H200 runs $4.47 to $10.60, B200 $7.69 to $14.24, and MI300X $2.59 to $6.00. Commitments and spot cut those numbers by 30% to 80%.
Figure 1: The normalisation pipeline used for this cloud GPU pricing benchmark, from raw price page to effective cost per useful token.
The pipeline in Figure 1 starts with the price as the provider publishes it, which might be per instance, per GPU, per node or per contract term. It then divides by the GPU count, converts reservation totals into hourly rates, records the pricing tier, and only then compares. The final two stages, utilisation and throughput, convert a sticker price into the number that actually hits your margin: cost per useful token or per training step.
Every row is normalised to dollars per GPU-hour, with the instance math shown
Hyperscalers sell eight-GPU virtual machines; neoclouds mostly quote per GPU. Comparing a $55.04 instance with a $4.39 GPU is meaningless until you divide. The rule used here is simple: per-GPU-hour equals instance hourly price divided by GPU count. An AWS p5.48xlarge at $55.04 holds eight H100s, so it costs $6.88 per GPU-hour.
Reservations need one more step. Azure’s retail prices API reports a reserved instance as a total for the whole term, not an hourly rate. A one-year ND96isr H100 v5 reservation is listed at $551,221. Dividing by 8,760 hours gives $62.92 per instance-hour, or $7.87 per GPU-hour. The three-year term, $1,134,310 over 26,280 hours, works out to $43.16 per instance-hour and $5.40 per GPU-hour.
Superchip systems break the eight-GPU assumption. Azure’s ND128isr NDR GB200 v6 and CoreWeave’s GB200 NVL72 instance each expose four Blackwell GPUs per virtual machine, so their hourly price is divided by four, not eight. Getting this wrong makes GB200 look half as expensive as it is.
Sources, date and scope
All prices were read on 14 November 2026 from official provider pages or official pricing APIs wherever possible. The scope is US regions, Linux, list prices, in US dollars, before tax. Where a provider does not publish a number, the cell says so rather than borrowing a third-party estimate.
Two sources need a disclosure. AWS on-demand and spot figures come from the SpareCores price tracker, which ingests the public AWS price list and showed observations dated 11 to 14 November 2026; AWS’s own on-demand page renders prices dynamically and could not be read directly. AWS Capacity Block rates come straight from the official Capacity Blocks pricing page. Azure figures come from the public Azure Retail Prices API, queried directly for the East US 2 region.
The neocloud set is CoreWeave, Lambda, Nebius, RunPod, DigitalOcean, Vultr and Crusoe. Oracle Cloud Infrastructure offers BM.GPU.H100.8, H200.8, B200.8, GB200 and MI300X.8 shapes, but its public pricing page displayed empty price cells when fetched, so OCI is listed as unverified rather than guessed.
The headline results table
The table below is the core of this cloud GPU pricing benchmark. Every figure is a list on-demand rate unless the tier column says otherwise.
| GPU | Provider and SKU | Tier | Published price | GPUs | $ per GPU-hour |
|---|---|---|---|---|---|
| H100 80GB | Azure ND96isr H100 v5, East US 2 | On-demand | $98.32 per VM-hr | 8 | 12.29 |
| H100 80GB | Google Cloud a3-megagpu-8g, us-central1 | On-demand | $93.40 per VM-hr | 8 | 11.68 |
| H100 80GB | Google Cloud a3-highgpu-8g, us-central1 | On-demand | $88.49 per VM-hr | 8 | 11.06 |
| H100 80GB | AWS p5.48xlarge, us-east-1 | On-demand | $55.04 per instance-hr | 8 | 6.88 |
| H100 80GB | CoreWeave HGX H100 | On-demand | $49.24 per instance-hr | 8 | 6.16 |
| H100 80GB | Lambda | On-demand, effective 16 Nov 2026 | $4.59 per GPU-hr | 1 | 4.59 |
| H100 80GB | Nebius HGX H100 | On-demand | $4.50 per GPU-hr | 1 | 4.50 |
| H100 80GB | DigitalOcean HGX H100 | On-demand | $4.41 per GPU-hr | 1 | 4.41 |
| H100 80GB | RunPod H100 SXM, Secure Cloud | On-demand | $4.39 per GPU-hr | 1 | 4.39 |
| H200 141GB | Azure ND96isr H200 v5, East US 2 | On-demand | $84.80 per VM-hr | 8 | 10.60 |
| H200 141GB | Google Cloud a3-ultragpu-8g | On-demand | $84.81 per VM-hr | 8 | 10.60 |
| H200 141GB | AWS p5en.48xlarge, us-east-1 | On-demand | $63.30 per instance-hr | 8 | 7.91 |
| H200 141GB | CoreWeave HGX H200 | On-demand | $50.44 per instance-hr | 8 | 6.31 |
| H200 141GB | Nebius HGX H200 | On-demand | $5.40 per GPU-hr | 1 | 5.40 |
| H200 141GB | RunPod H200, Secure Cloud | On-demand | $5.29 per GPU-hr | 1 | 5.29 |
| H200 141GB | DigitalOcean HGX H200 | On-demand | $4.47 per GPU-hr | 1 | 4.47 |
| B200 180GB | AWS p6-b200.48xlarge, us-east-1 | On-demand | $113.93 per instance-hr | 8 | 14.24 |
| B200 180GB | CoreWeave HGX B200 | On-demand | $68.80 per instance-hr | 8 | 8.60 |
| B200 180GB | Nebius HGX B200 | On-demand | $8.50 per GPU-hr | 1 | 8.50 |
| B200 180GB | RunPod B200 | On-demand | $7.99 per GPU-hr | 1 | 7.99 |
| B200 180GB | Lambda | On-demand, effective 16 Nov 2026 | $7.69 per GPU-hr | 1 | 7.69 |
| B200 180GB | Google Cloud a4-highgpu-8g | On-demand | Not offered on-demand, N/A on page | 8 | n/a |
| GB200 | Azure ND128isr NDR GB200 v6 | On-demand | $108.16 per VM-hr | 4 | 27.04 |
| GB200 | CoreWeave GB200 NVL72 instance | On-demand | $42.00 per instance-hr | 4 | 10.50 |
| MI300X 192GB | Azure ND96isr MI300X v5, East US 2 | On-demand | $48.00 per VM-hr | 8 | 6.00 |
| MI300X 192GB | DigitalOcean | On-demand | $2.59 per GPU-hr | 1 | 2.59 |
| MI300X 192GB | Oracle BM.GPU.MI300X.8 | On-demand | Not displayed, unverified | 8 | n/a |
Sources, all read 14 November 2026: Azure Retail Prices API; Google Cloud accelerator-optimized pricing page; AWS figures via the SpareCores tracker of the AWS public price list; CoreWeave, Lambda, Nebius, RunPod and DigitalOcean official pricing pages. Lambda’s page states its instance prices take effect 16 November 2026 at 00:00 UTC.
Three patterns jump out. First, AWS is now the cheapest hyperscaler for Hopper by a wide margin, a direct legacy of the June 2025 cut that Azure’s list price never matched. Azure’s ND H100 v5 retail rate still carries an effective date of December 2023 in the API.
Second, neocloud GPU pricing clusters tightly. Lambda, Nebius, DigitalOcean and RunPod all sit between $4.39 and $4.59 for an H100, which looks like a competitive equilibrium rather than coincidence. CoreWeave sits higher at $6.16, but its price includes a large node with 128 vCPUs, 2 TB of RAM and InfiniBand fabric.
Third, Blackwell carries a hyperscaler premium larger than Hopper did. AWS’s B200 on-demand rate of $14.24 is 1.66x CoreWeave’s $8.60, and about 1.85x Lambda’s upcoming $7.69. The Azure GB200 figure of $27.04 per GPU-hour is an outlier that is worth checking with your account team before using it in a model; it is what the retail API returns, but very little GB200 capacity is likely sold at that list rate.
Deeper Analysis: Commitments, Spot, Price per GB of HBM and Cost per Million Tokens
On-demand list prices are the ceiling, not the market. Most serious GPU spend runs on a commitment, a capacity reservation, or interruptible capacity. The second layer of this benchmark looks at those tiers, then normalises by memory and compute, then turns dollars per hour into dollars per token.
Figure 2: The pricing ladder. Each rung trades flexibility for a lower effective rate per GPU-hour.
Figure 2 orders the tiers by how much flexibility you surrender. Hyperscaler on-demand sits at the top. Below it come short-term reservations such as AWS Capacity Blocks and Google’s Dynamic Workload Scheduler, then neocloud on-demand, then one-year and three-year commitments, and finally spot or preemptible capacity, which is cheapest because the provider can take it back.
Commitment and spot tiers, normalised
| GPU | Provider and tier | Published rate | $ per GPU-hour | vs on-demand |
|---|---|---|---|---|
| H100 | AWS Capacity Block, p5.48xlarge, us-east-1 | $47.757 per instance-hr | 5.97 | -13% |
| H100 | AWS spot, p5.48xlarge, us-east-1 | $31.16 per instance-hr | 3.90 | -43% |
| H100 | Azure 1-year reserved, ND96isr H100 v5 | $551,221 per term | 7.87 | -36% |
| H100 | Azure 3-year reserved, ND96isr H100 v5 | $1,134,310 per term | 5.40 | -56% |
| H100 | Azure spot, ND96isr H100 v5 | $18.17 per VM-hr | 2.27 | -82% |
| H100 | Google DWS Flex-start, a3-highgpu-8g | $38.32 per VM-hr | 4.79 | -57% |
| H100 | Google 3-year resource CUD, a3-highgpu-8g | $38.86 per VM-hr | 4.86 | -56% |
| H100 | Google spot, a3-highgpu-8g | $52.70 per VM-hr | 6.59 | -40% |
| H100 | DigitalOcean 12-month reserved | $3.26 per GPU-hr | 3.26 | -26% |
| H100 | RunPod Community Cloud, H100 SXM | from $2.69 per GPU-hr | 2.69 | -39% |
| H200 | AWS Capacity Block, p5en.48xlarge | $63.158 per instance-hr | 7.90 | 0% |
| H200 | AWS spot, p5en.48xlarge, us-east-1 | $39.47 per instance-hr | 4.93 | -38% |
| H200 | Azure 3-year reserved, ND96isr H200 v5 | $1,109,592 per term | 5.28 | -50% |
| H200 | Google 3-year resource CUD, a3-ultragpu-8g | $37.21 per VM-hr | 4.65 | -56% |
| B200 | AWS Capacity Block, p6-b200.48xlarge | $113.666 per instance-hr | 14.21 | 0% |
| B200 | AWS spot, p6-b200.48xlarge, us-east-1 | $69.03 per instance-hr | 8.63 | -39% |
| B200 | Google DWS Flex-start, a4-highgpu-8g | $64.44 per VM-hr | 8.06 | n/a |
| B200 | Google spot, a4-highgpu-8g | $43.60 per VM-hr | 5.45 | n/a |
| B200 | Google 3-year resource CUD, a4-highgpu-8g | $56.71 per VM-hr | 7.09 | n/a |
| B200 | Lambda 1-Click Cluster, 256+ GPUs | $8.87 per GPU-hr | 8.87 | +15% vs Lambda instance |
| B300 | AWS Capacity Block, p6-b300.48xlarge | $129.168 per instance-hr | 16.15 | n/a |
| B300 | DigitalOcean 12-month reserved | $7.94 per GPU-hr | 7.94 | n/a |
| GB200 | AWS Capacity Block, p6e-gb200 UltraServer | $761.904 per 72-GPU UltraServer-hr | 10.58 | n/a |
| MI300X | Azure 3-year reserved, ND96isr MI300X v5 | $553,772 per term | 2.63 | -56% |
| MI300X | Azure spot, ND96isr MI300X v5 | $8.87 per VM-hr | 1.11 | -82% |
| MI300X | DigitalOcean 12-month reserved | $1.91 per GPU-hr | 1.91 | -26% |
| MI300X | Vultr preemptible | $1.85 per GPU-hr | 1.85 | n/a |
All read 14 November 2026. Spot rates move daily or faster; the AWS and Azure spot rows are point observations, not quotes. “vs on-demand” compares against the same provider’s on-demand rate where one exists.
A few rows deserve a closer look. AWS Capacity Blocks for H200 and B200 cost the same as, or slightly less than, on-demand. That is the January 2026 increase showing through: a Capacity Block is now priced as guaranteed availability, not as a discount. For H100, the Capacity Block still sits 13% below on-demand.
Google’s Dynamic Workload Scheduler Flex-start is the sleeper deal among hyperscalers. At $4.79 per GPU-hour, an a3-highgpu-8g in Flex-start mode costs about the same as a neocloud on-demand H100, with the trade-off that Flex-start queues your request until capacity frees up. For batch training and offline inference, that wait often costs nothing.
Lambda’s 1-Click Cluster pricing runs above its single-instance rate. That inversion is rational: a cluster bundles a guaranteed InfiniBand fabric, co-located nodes and a fixed term of two weeks or more. You are buying topology, not just GPUs.
Vultr’s pricing page also lists HGX H100 at $1.99 per GPU-hour on a 24-month contract and HGX B200 at $1.99 on a 48-month contract. Those figures are far below every other row in this benchmark and are presented on the page as “starting at” contract rates, so treat them as unverified headline pricing until a quote confirms them for your volume and term.
Price per GB of HBM and per dense FP8 petaflop
Dollars per GPU-hour hides the fact that the GPUs are not equal. Large-model inference is usually bound by memory capacity and bandwidth, while training is usually bound by matrix throughput. Normalising by both changes the ranking.
Vendor specifications give the denominators. NVIDIA’s H100 SXM carries 80 GB of HBM3 at about 3.35 TB/s; the H200 carries 141 GB of HBM3e at about 4.8 TB/s with the same compute. NVIDIA’s DGX B200 page lists 1,440 GB and 64 TB/s across eight GPUs, which is 180 GB and 8 TB/s per GPU, and 72 PFLOPS of sparse FP8, or about 4.5 PFLOPS dense per GPU. AMD lists MI300X at 192 GB of HBM3, 5.3 TB/s and 2.61 PFLOPS of dense FP8.
| GPU and price point | $ per GPU-hr | HBM GB | $ per HBM GB-hr | Dense FP8 PFLOPS | $ per dense FP8 PFLOP-hr |
|---|---|---|---|---|---|
| H100, AWS on-demand | 6.88 | 80 | 0.086 | about 1.98 | 3.47 |
| H100, neocloud median-ish | 4.50 | 80 | 0.056 | about 1.98 | 2.27 |
| H200, AWS on-demand | 7.91 | 141 | 0.056 | about 1.98 | 4.00 |
| H200, Nebius on-demand | 5.40 | 141 | 0.038 | about 1.98 | 2.73 |
| B200, AWS on-demand | 14.24 | 180 | 0.079 | about 4.5 | 3.16 |
| B200, Lambda on-demand | 7.69 | 180 | 0.043 | about 4.5 | 1.71 |
| MI300X, Azure on-demand | 6.00 | 192 | 0.031 | 2.61 | 2.30 |
| MI300X, DigitalOcean on-demand | 2.59 | 192 | 0.013 | 2.61 | 0.99 |
Peak specs are vendor datasheet numbers and are never reached in practice. FP8 dense figures for H100 and H200 are half of NVIDIA’s published sparse figures. Treat the last column as a ceiling-based ratio, not achieved performance.
The memory view explains the MI300X’s niche. At $0.013 to $0.031 per GB-hour of HBM, it is the cheapest way to hold a large model’s weights and KV cache in GPU memory. A 70-billion-parameter model in FP8 needs about 70 GB for weights alone, which fits on one MI300X with room for cache but forces tensor parallelism across two H100s.
The compute view explains why B200 wins for training at neocloud prices despite the higher sticker. At $7.69 per GPU-hour, a B200 delivers a dense FP8 petaflop-hour for about $1.71, against $2.27 on a $4.50 H100. At AWS on-demand prices the advantage shrinks to almost nothing. The B200 cloud pricing gap between hyperscaler and neocloud is therefore more important than the generational gap.
Peak FLOPS ratios flatter MI300X. Achieved throughput depends heavily on the software stack, and our ROCm versus CUDA analysis for MI300X and MI350 explains where kernels and serving engines still lag. Use the MI300X column as an upper bound on value, then discount it by your measured efficiency gap.
From dollars per hour to dollars per million tokens
The number that reaches your income statement is cost per unit of work. For inference that is cost per million output tokens; for training it is cost per step or per billion training tokens. Figure 3 shows the chain.
Figure 3: How an hourly GPU price becomes a cost per million tokens. Utilisation and throughput matter as much as the rate.
The formula is short. Cost per million tokens equals the hourly price for the deployment divided by the tokens it actually produces per hour, times one million. Tokens per hour equals aggregate throughput in tokens per second times 3,600 times utilisation. Utilisation is the fraction of paid hours in which the GPUs serve real traffic at the throughput you measured.
# Illustrative cost-per-million-tokens model. Throughput numbers are ASSUMPTIONS,
# not measurements. Replace them with your own benchmark results.
def cost_per_mtok(gpu_hour_price, gpus, tokens_per_sec, utilisation):
hourly = gpu_hour_price * gpus
tokens_per_hour = tokens_per_sec * 3600 * utilisation
return hourly / tokens_per_hour * 1_000_000
scenarios = {
# name: (price per GPU-hour, GPUs, assumed aggregate output tok/s, utilisation)
"H100 AWS on-demand": (6.88, 8, 10_000, 0.60),
"H100 neocloud": (4.50, 8, 10_000, 0.60),
"H100 Azure 3-yr RI": (5.40, 8, 10_000, 0.85),
"B200 AWS on-demand": (14.24, 8, 25_000, 0.60),
"B200 neocloud": (7.69, 8, 25_000, 0.60),
"MI300X DigitalOcean": (2.59, 8, 8_000, 0.60),
}
for name, args in scenarios.items():
print(f"{name:22s} ${cost_per_mtok(*args):.3f} per million output tokens")
Running that script prints the following. These outputs are illustrative, because the throughput inputs are assumptions chosen to show the mechanics, not measured results.
H100 AWS on-demand $2.548 per million output tokens
H100 neocloud $1.667 per million output tokens
H100 Azure 3-yr RI $1.412 per million output tokens
B200 AWS on-demand $2.110 per million output tokens
B200 neocloud $1.139 per million output tokens
MI300X DigitalOcean $1.199 per million output tokens
Three lessons survive any choice of inputs. First, utilisation is a multiplier on every price: the Azure three-year reservation beats the cheaper neocloud H100 in this example only because a committed fleet is assumed to run at 85% rather than 60%. If your traffic cannot fill a reservation, the discount is fiction.
Second, the B200 rows assume 2.5x the H100 throughput per GPU. Real ratios depend on model, precision, batch size and serving engine, and published results vary widely. Our vLLM, SGLang and TensorRT-LLM benchmark on H100 shows how much the engine alone moves tokens per second on identical hardware; that spread is often larger than the price spread between providers.
Third, throughput must be measured at your latency target. A fleet that hits 10,000 tokens per second at a 2-second time to first token is not the same product as one that hits it at 200 milliseconds. Our guide to LLM inference SLOs, goodput and capacity planning explains how to measure goodput, the throughput that actually meets the SLO, which is the only honest denominator.
Methodology caveats
This is a list-price benchmark, and list prices are a starting point. Five caveats apply to every number above.
Regional variation is large. The same AWS p5.48xlarge costs $55.04 in us-east-1 and $92.47 in São Paulo, according to the tracker data. Azure’s H100 VM costs $98.32 in East US 2 and more than $127 in West US. Pick the region before comparing providers.
List prices are not transaction prices. Enterprise discount programmes, private pricing agreements and marketplace credits routinely move hyperscaler effective rates well below list, and neocloud contracts for hundreds of GPUs are negotiated. None of that is public, and none of it is reflected here.
The hardware is not identical even when the GPU is. A hyperscaler eight-GPU VM includes dozens of CPU cores, terabytes of RAM, local NVMe, and 3,200 Gbps-class fabric. A neocloud single-GPU instance may include far less. Compare node specifications before concluding one is cheaper.
Availability is not priced in. A $4.39 rate is worth nothing if the provider has no capacity in your region this week. Spot and community-cloud figures are “from” prices that may not be obtainable when you need them.
Finally, prices are moving. Lambda’s new rates take effect on 16 November, Crusoe’s dedicated-endpoint rates rise on 1 December, and spot prices change daily. Treat this as a dated snapshot and rerun the method before you sign anything.
The MI300X row deserves its own warning. It assumes 8,000 tokens per second per eight-GPU node, which is a placeholder, not a measured figure. If your serving stack reaches only half that on MI300X for your model, its cost per million tokens doubles to about $2.40, which is worse than the neocloud H100. Price per hour is the easy part; the measured throughput is where AMD’s value either holds or disappears.
Trade-offs, Gotchas, and What Goes Wrong
The cheapest rate in a table is rarely the cheapest outcome. Most cost overruns on GPU clouds come from things the hourly price does not show.
Figure 4: The GPU hour is only the visible layer of total cost. Storage, data movement, idle capacity and operations sit underneath.
Egress, storage and networking change the ranking
Figure 4 stacks the cost layers that sit under the GPU line item. Data egress is the classic trap. AWS includes 100 GB of free data transfer out to the internet per month and bills tiered per-GB rates above that; the first paid tier is widely quoted at $0.09 per GB, which you should confirm on the AWS data-transfer pricing page for your region. Moving a 10 TB checkpoint and dataset set out once costs on the order of $900 at that rate.
CoreWeave’s pricing page states that egress, ingress, transfer and IOPS are free, and lists AI Object Storage at $0.06 per GB-month for hot data and Distributed File Storage at $0.07 per GB-month. For a team that moves checkpoints between clouds or serves large media outputs, zero egress can outweigh a higher GPU rate. The reverse also holds: a hyperscaler GPU that sits next to your data lake avoids the egress entirely.
Interconnect is the other hidden variable. Multi-node training needs a non-blocking InfiniBand or RoCE fabric with GPUDirect RDMA. A cheap single-node rate tells you nothing about whether the provider can give you 64 nodes on one fabric. Our explainer on NVLink, InfiniBand and RoCE for collectives covers why all-reduce efficiency can swing effective training cost by tens of percent.
Idle time is the biggest line item nobody budgets
A GPU you pay for and do not use costs the full rate. On-demand fleets left running overnight, reservations sized for peak, and nodes waiting on a data loader all burn money. Autoscaling closes part of the gap; our comparison of Karpenter and Cluster Autoscaler for GPU nodes shows how provisioning latency and bin-packing decide how quickly idle GPUs disappear.
Sharing helps for small models. MIG partitions and time-slicing let several low-traffic models share one H100, which our guide to Kubernetes GPU sharing with MIG, time-slicing and MPS walks through. A 4x improvement in utilisation beats any provider switch.
Spot and preemptible capacity bite back
Spot rates of $2.27 per H100-hour on Azure or $1.11 per MI300X-hour look extraordinary. They are cheap because the provider reclaims them, sometimes with as little as 30 seconds’ notice on some clouds and two minutes on AWS. Training on spot only works with frequent checkpointing and automatic restart, which our guide to spot GPU training with checkpointing and preemption handling details. Lost progress and restart overhead can erase half the discount.
Commitments create stranded assets
A three-year reservation at 56% off is a bet that you will still want that GPU in 2029. Blackwell already offers about 2.3x the dense FP8 throughput of Hopper per GPU, and NVIDIA has announced Rubin as the next generation. A team locked into three years of H100 at $5.40 per GPU-hour may find that a B200 on-demand delivers the same work for less within 18 months. Prefer one-year terms or convertible commitments for anything but stable inference baselines.
Anti-patterns to avoid
Comparing per-instance and per-GPU prices without dividing is the most common error in vendor comparisons. Assuming every H100 is equal is the second: PCIe H100s at $2.89 on RunPod are not SXM H100s, and they lack NVLink bandwidth for tensor parallelism. The third is optimising the GPU line while ignoring the 20% to 40% of the bill that storage, networking and support plans can represent at scale; that range is an estimate from typical architectures, not a measured benchmark.
Practical Recommendations
Start from the workload, not the price table. Inference, fine-tuning and pre-training have different sensitivities to price, availability, interconnect and interruption. The decision guide below maps the common cases to the tier that usually wins on cost per unit of work.
| Workload | Best-value tier in Nov 2026 | Why | Watch out for |
|---|---|---|---|
| Prototyping and evaluation | Neocloud on-demand H100 or H200, $4.39 to $5.40 | No commitment, per-second or per-minute billing | Capacity gaps in popular regions |
| Steady production inference | 1-year commitment or 12-month reserved, or 3-year if model stable | Utilisation above 80% justifies commitment | Stranded Hopper capacity as Blackwell scales |
| Bursty or batch inference | Spot, Google Flex-start, or autoscaled on-demand | Interruptions tolerable, queues acceptable | Restart overhead, quota limits |
| Memory-bound large models | MI300X or H200 | Lowest $ per GB of HBM | ROCm maturity for your serving engine |
| Multi-node training | Neocloud cluster or hyperscaler Capacity Block, B200 | Guaranteed fabric, best $ per FP8 PFLOP at neocloud rates | Contract terms, fabric topology |
| Data-heavy or regulated work | Hyperscaler in the same region as your data | Avoids egress and compliance work | Highest list rates |
Measure before you buy. Run your own model, at your own latency SLO, on two or three candidate SKUs for a day each, then compute cost per million tokens with the formula above. A day of benchmarking costs a few hundred dollars and routinely saves more than that every week.
Price the whole stack. Ask every provider for egress, storage, and interconnect pricing in writing, and model one month of realistic data movement. Our broader guide to AI inference cost optimisation covers the software levers, including quantisation, batching and caching, that compound with a good price.
Keep optionality. Blend a committed baseline with on-demand or spot burst capacity, and avoid committing more than your lowest month of utilisation. Re-benchmark quarterly; the market moved 12% in the last year.
- Normalise every quote to $ per GPU-hour, dividing by GPU count and term hours.
- Record region, tier, node specs and date next to each number.
- Measure goodput at your SLO on at least two GPU types.
- Compute cost per million tokens or per training step, not per hour.
- Add egress, storage and fabric costs before ranking providers.
- Cap commitments at your trough utilisation and prefer 1-year terms.
- Rerun this comparison every quarter.
Frequently Asked Questions
How much does an H100 cost per hour in the cloud in November 2026?
List prices on 14 November 2026 range from about $4.39 to $12.29 per GPU-hour on-demand in US regions. Neoclouds such as RunPod, DigitalOcean, Nebius and Lambda cluster between $4.39 and $4.59. CoreWeave lists $6.16, AWS p5.48xlarge works out to $6.88, Google Cloud’s a3-highgpu-8g to $11.06, and Azure’s ND H100 v5 to $12.29. Spot, community-cloud and committed rates can fall to between roughly $2.27 and $5.40, subject to availability and contract terms.
What is the B200 cloud pricing per GPU-hour?
On-demand B200 pricing ranged from $7.69 at Lambda, effective 16 November 2026, to $14.24 on AWS p6-b200.48xlarge in us-east-1. RunPod lists $7.99, Nebius $8.50 and CoreWeave $8.60. Google Cloud does not list an on-demand rate for a4-highgpu-8g, but offers Flex-start at about $8.06 and spot at about $5.45 per GPU-hour. AWS Capacity Blocks for B200 cost $14.21 per GPU-hour, nearly identical to on-demand.
Why are hyperscalers more expensive than neoclouds for GPUs?
Hyperscaler GPU VMs bundle more hardware, including large CPU and memory allocations, local NVMe and high-bandwidth fabric, plus integration with managed services, compliance programmes and global regions. They also price for enterprises that receive negotiated discounts off list. Neoclouds specialise in GPU capacity, run leaner software stacks and compete hard on headline rates. The gap is real at list prices, but private pricing and avoided egress can narrow it substantially for large customers.
Is the MI300X cheaper than the H100 for inference?
Per hour and per gigabyte of memory, usually yes. MI300X lists at $2.59 per GPU-hour on DigitalOcean and $6.00 on Azure, and carries 192 GB of HBM3 against the H100’s 80 GB. That makes it the cheapest way to hold large models in memory. Whether it is cheaper per token depends on achieved throughput, which depends on ROCm support in your serving engine. Benchmark your exact model before switching.
Are GPU cloud prices going up or down in 2026?
Up, on balance. The GetDeploying index reported median GPU rental prices up 12% year on year in November 2026, with H100 up 31%. AWS raised Capacity Block prices about 15% in January 2026, CoreWeave’s CFO described increases across generations, and Crusoe has published higher rates from 1 December 2026. New Blackwell supply may push prices down later, but that had not happened by mid-November.
Should I use spot GPUs for LLM training?
Yes, if your training loop can checkpoint frequently and restart automatically. Spot H100 capacity listed at about $2.27 per GPU-hour on Azure and $3.90 on AWS during this snapshot, a 43% to 82% discount. The cost is interruptions, lost progress since the last checkpoint and restart overhead. For large multi-node jobs, a single reclaimed node can stall the whole run, so many teams use spot only for smaller jobs, evaluations and batch inference.
Further Reading
- Karpenter vs Cluster Autoscaler for GPU node scaling — how autoscaling decisions turn idle GPU hours into savings.
- vLLM vs SGLang vs TensorRT-LLM on H100 — the throughput half of the cost-per-token equation.
- Hyperscaler capex and AI compute economics — the supply-side forces behind 2026 GPU pricing.
- AWS: up to 45% price reduction for EC2 NVIDIA GPU instances — the official June 2025 cut that set today’s P5 rates.
- Google Cloud accelerator-optimized pricing — A3 and A4 on-demand, spot, DWS and CUD rates.
By Riju — about
