StarCoder2 15B
bigcode/starcoder2-15b
Parameters
15B
Context
16K
Downloads
3.8K
Likes
674
Architecture
transformer
GPU Requirements
VRAM / RAM / disk estimates by quantization and usage scenario
| Quantization | Minimum | Recommended | Production |
|---|---|---|---|
| FP1616bit | 40.3 GB RAM 60.4 · Disk 82.6 | 57.7 GB RAM 86.5 · Disk 108.6 | 156.2 GB RAM 234.2 · Disk 147.6 |
| FP88bit | 22.6 GB RAM 33.9 · Disk 60.8 | 39.2 GB RAM 58.8 · Disk 86.8 | 136.9 GB RAM 205.3 · Disk 125.8 |
| INT88bit | 22.6 GB RAM 33.9 · Disk 60.8 | 39.2 GB RAM 58.8 · Disk 86.8 | 136.9 GB RAM 205.3 · Disk 125.8 |
| AWQ4bit | 14.1 GB RAM 32.0 · Disk 50.3 | 30.3 GB RAM 45.4 · Disk 76.3 | 127.6 GB RAM 191.4 · Disk 115.3 |
| GPTQ4bit | 14.2 GB RAM 32.0 · Disk 50.4 | 30.4 GB RAM 45.6 · Disk 76.4 | 127.7 GB RAM 191.6 · Disk 115.4 |
| GGUF4bit | 14.5 GB RAM 32.0 · Disk 50.7 | 30.6 GB RAM 46.0 · Disk 76.7 | 128.0 GB RAM 191.9 · Disk 115.7 |
GPU Recommendations
1× H100 SXM
80 GB VRAM
≈ 462.1 tok/s
Effective production throughput ≈ 323.5 tok/s
score 0.5905
2× Tesla V100
32 GB VRAM
≈ 0 tok/s (N/A — benchmark unavailable)
Effective production throughput ≈ 0 tok/s
score 0.5943
8× H200 SXM
1128 GB VRAM
≈ 2118.6 tok/s
Effective production throughput ≈ 1483 tok/s
score 0.2197
1× RTX A6000
48 GB VRAM
≈ 105.9 tok/s
Effective production throughput ≈ 74.1 tok/s
score 0.6593
💬 Community — real deployment experience
0 Articles · 0 Benchmarks
GPUs that fit (single card)
RTX A6000
48 GB VRAM
$210
/mo · vast
RTX 5090
32 GB VRAM
$236
/mo · vast
L40S
48 GB VRAM
$341
/mo · vast
RTX PRO 6000 WS
48 GB VRAM
$482
/mo · vast
RTX PRO 6000 S
48 GB VRAM
$633
/mo · vast