Mistral Medium 3.5 128B
mistralai/Mistral-Medium-3.5-128B
Parameters
128B
Context
256K
Downloads
80.7K
Likes
418
Architecture
transformer
GPU Requirements
VRAM / RAM / disk estimates by quantization and usage scenario
| Quantization | Minimum | Recommended | Production |
|---|---|---|---|
| FP1616bit | 305.2 GB RAM 457.8 · Disk 410.9 | 321.1 GB RAM 481.6 · Disk 436.9 | 350.6 GB RAM 525.9 · Disk 475.9 |
| FP88bit | 154.4 GB RAM 231.6 · Disk 225.0 | 163.4 GB RAM 245.1 · Disk 251.0 | 186.1 GB RAM 279.2 · Disk 290.0 |
| INT88bit | 154.4 GB RAM 231.6 · Disk 225.0 | 163.4 GB RAM 245.1 · Disk 251.0 | 186.1 GB RAM 279.2 · Disk 290.0 |
| AWQ4bit | 81.8 GB RAM 122.7 · Disk 135.5 | 87.5 GB RAM 131.3 · Disk 161.5 | 106.9 GB RAM 160.4 · Disk 200.5 |
| GPTQ4bit | 82.7 GB RAM 124.1 · Disk 136.6 | 88.5 GB RAM 132.8 · Disk 162.6 | 108.0 GB RAM 162.0 · Disk 201.6 |
| GGUF4bit | 84.6 GB RAM 126.9 · Disk 139.0 | 90.5 GB RAM 135.7 · Disk 165.0 | 110.0 GB RAM 165.0 · Disk 204.0 |
GPU Recommendations
Best ValueBEST
1× H200 SXM
141 GB VRAM
$2,555/mo
≈ 77.6 tok/s
Effective production throughput ≈ 54.3 tok/s
score 0.6366
Cheapest
8× Tesla V100
128 GB VRAM
$162/mo
≈ 0 tok/s (N/A — benchmark unavailable)
Effective production throughput ≈ 0 tok/s
score 0.539
Performance
8× H200 SXM
1128 GB VRAM
$20,440/mo
≈ 248.4 tok/s
Effective production throughput ≈ 173.9 tok/s
score 0.2197
Min Complexity
2× RTX A6000
96 GB VRAM
$420/mo
≈ 19.9 tok/s
Effective production throughput ≈ 13.9 tok/s
score 0.6044
💬 Community — real deployment experience
0 Articles · 0 Benchmarks
GPUs that fit (single card)
H200 SXM
141 GB VRAM
$2,555
/mo · lambda
B200
180 GB VRAM
$3,013
/mo · vast
B200 SXM
180 GB VRAM
—