Qwen2-VL 72B
Qwen/Qwen2-VL-72B-Instruct
Parameters
72B
Context
32K
Downloads
60.3K
Likes
311
Architecture
transformer
GPU Requirements
VRAM / RAM / disk estimates by quantization and usage scenario
| Quantization | Minimum | Recommended | Production |
|---|---|---|---|
| FP1616bit | 173.2 GB RAM 259.8 · Disk 248.2 | 183.1 GB RAM 274.7 · Disk 274.2 | 206.7 GB RAM 310.0 · Disk 313.2 |
| FP88bit | 88.4 GB RAM 132.6 · Disk 143.6 | 94.4 GB RAM 141.7 · Disk 169.6 | 114.1 GB RAM 171.2 · Disk 208.6 |
| INT88bit | 88.4 GB RAM 132.6 · Disk 143.6 | 94.4 GB RAM 141.7 · Disk 169.6 | 114.1 GB RAM 171.2 · Disk 208.6 |
| AWQ4bit | 47.6 GB RAM 71.4 · Disk 93.3 | 51.8 GB RAM 77.6 · Disk 119.3 | 69.6 GB RAM 104.4 · Disk 158.3 |
| GPTQ4bit | 48.1 GB RAM 72.2 · Disk 93.9 | 52.3 GB RAM 78.5 · Disk 119.9 | 70.2 GB RAM 105.3 · Disk 158.9 |
| GGUF4bit | 49.2 GB RAM 73.8 · Disk 95.2 | 53.4 GB RAM 80.1 · Disk 121.2 | 71.3 GB RAM 107.0 · Disk 160.2 |
GPU Recommendations
1× H100 SXM
80 GB VRAM
≈ 96.3 tok/s
Effective production throughput ≈ 67.4 tok/s
score 0.6572
4× RTX 3090
96 GB VRAM
≈ 64.6 tok/s
Effective production throughput ≈ 45.2 tok/s
score 0.5524
8× H200 SXM
1128 GB VRAM
≈ 441.6 tok/s
Effective production throughput ≈ 309.1 tok/s
score 0.2197
1× A100 PCIE
80 GB VRAM
≈ 0 tok/s (N/A — benchmark unavailable)
Effective production throughput ≈ 0 tok/s
score 0.6544
💬 Community — real deployment experience
0 Articles · 0 Benchmarks
GPUs that fit (single card)
A100 PCIE
80 GB VRAM
$390
/mo · vast
A100 SXM4
80 GB VRAM
$440
/mo · vast
A100 80GB
80 GB VRAM
$876
/mo · vast
H100 SXM
80 GB VRAM
$1,404
/mo · vast
H200 NVL
141 GB VRAM
$1,608
/mo · vast