✦ WORKLOAD RECOMMENDATIONS

Best GPU for Machine Learning Training in 2026: Price vs Performance Guide

Why GPU Selection Matters More Than Ever in 2026

The GPU market has fractured. You no longer simply pick "the best GPU" — you pick the right GPU for your workload, your budget, and your interruption tolerance. An H100 costs $46/hr on AWS but $1.47/hr on Vast.ai. An A100 that's $1.21/hr on CoreWeave can be $0.43/hr on Vast.ai on any given day. That variance changes which GPU is actually the right choice.

Three specs determine GPU suitability for ML training: VRAM (determines max batch size and model size), FP16 TFLOPS (raw throughput for matrix operations), and interconnect bandwidth (NVLink/NVSwitch for multi-GPU scaling). Price matters, but it's the last of these three to evaluate.

GPU Tier List Based on Real Pricing Data

The table below shows 30-day average spot prices per GPU per provider. Use these to see where the actual price floor is — not the list price, but what you're likely to pay on a good day.

Provider Recommendations by Use Case

Price is one variable; availability, stability, and setup complexity are others. Here's the practical recommendation per use case based on 30-day data patterns:

Understanding $/TFLOP: The Real Efficiency Metric

Price per hour is not the whole story. A GPU that costs twice as much but delivers 3× the throughput has better $/TFLOP — the metric that actually measures your cost efficiency per unit of compute. Here's how the tier compares:

• RTX 4090 (budget): $0.13–$0.34/hr for 330 FP16 TFLOPS. Exceptional $/TFLOP at current spot prices. Best for fine-tuning 7B–13B models, experiments, and any workload where interruption is acceptable.
• L40S (mid): $0.79/hr on RunPod, $1.84/hr on CoreWeave. 730 FP16 TFLOPS. Strong value for inference and medium-scale training without requiring H100-scale memory bandwidth.
• A100 (mid): $0.43–$1.29/hr across Vast.ai, RunPod, Lambda, CoreWeave. 1,248 FP16 TFLOPS. The production workhorse — excellent $/TFLOP, widely available, mature tooling.
• H100 (high): $1.47–$2.59/hr on specialists (Vast.ai, CoreWeave, Lambda, RunPod). 1,979 FP16 TFLOPS. Highest raw throughput. 2.5× the FP16 of A100. Essential for pretraining 70B+ models or multi-node training where time-to-train matters.

When Spot Pricing Makes Sense vs. Reserved/On-Demand

Hybrid strategy: Run your training on spot with aggressive checkpointing (save every 100–500 steps depending on step duration). Keep a single on-demand or reserved instance for your primary model serving endpoint. This gives you spot economics for training while protecting your production inference SLA.

For large-scale pretraining runs where every hour of wall time translates to days of schedule delay, reserved capacity on CoreWeave or Lambda Cloud (1.2–2.6× cheaper than AWS/GCP on H100) is worth the premium. The consistency of pricing matters more than the absolute floor when you're burning 100+ GPUs per run.

Multi-GPU Training: NVLink Changes the Math

If you're doing multi-GPU training, interconnect matters enormously. PCIe bandwidth between GPUs creates a bottleneck that NVLink eliminates. Only specialist providers (CoreWeave, Lambda Cloud, and the hyperscalers) offer NVLink-connected H100 and A100 nodes. Vast.ai and RunPod primarily offer PCIe-connected GPUs.

For pretraining 70B+ models, you need NVLink or NVSwitch to maintain efficient tensor parallelism across 8+ GPUs. The NVLink-connected tier on CoreWeave (8×H100 SXM, ~$16–$20/hr) is the right choice. Trying to replicate this on PCIe-connected GPUs will produce 30–50% lower effective throughput due to inter-GPU communication overhead.

For fine-tuning 7B–30B models, a single A100 or H100 with 80GB VRAM is sufficient for most use cases. Multi-GPU fine-tuning (FSDP, DeepSpeed ZeRO-3) is only necessary when the model doesn't fit in a single 80GB card — which is becoming rarer as quantization techniques improve.

Quick Decision Guide

• Fitting a 7B model at 4-bit? → RTX 4090 on Vast.ai ($0.01–$0.34/hr). Cheap, fast enough.
• Fine-tuning 7B–30B at FP16? → A100 80GB on RunPod or Vast.ai ($0.43–$1.22/hr). Best balance of VRAM and $/hr.
• Pretraining 70B+ from scratch? → H100 SXM with NVLink on CoreWeave ($2.06/hr). Reserve for consistent runs.
• Running large inference batches? → L40S on RunPod ($0.79/hr). High throughput, lower cost than A100 for inference-scale work.
• Enterprise workload with SLA? → A100 or H100 on CoreWeave or Lambda Cloud. Stability and support justify the premium.

Methodology

All pricing data in this article comes from RoofRun's continuous polling of provider APIs and pricing pages every 30 minutes. Prices shown are spot instance rates — on-demand pricing is typically 2–4× higher. The "current spot" column reflects the most recent available snapshot per GPU/provider; the "30-day avg" column is the mean across the trailing 30 days. GPU TFLOPS are NVIDIA-published FP16 Tensor Core specs. $/TFLOP calculated as average spot price divided by FP16 TFLOPS. Raw data available via the public JSON API.