§01·spec · /gpus
RTX 3060 Ti
nvidia30 series8GB VRAM
§02·models that run on this GPU
11 total| Model | Vertical | Best speed | Min VRAM | Works | Benchmarks | |
|---|---|---|---|---|---|---|
| llama2 7b | llm | 73.07tokens/s | 8GB | ✓ | 1 | check ↗ |
| llava 7b | multimodal | 72tokens/s | 8GB | ✓ | 1 | check ↗ |
| wizardlm2 7b | llm | 70.79tokens/s | 8GB | ✓ | 1 | check ↗ |
| qwen2 7b | llm | 63.73tokens/s | 8GB | ✓ | 1 | check ↗ |
| Qwen2.5 7B | llm | 58.13tokens/s | 8GB | ✓ | 1 | check ↗ |
| Llama 3.1 8B | llm | 57.34tokens/s | 8GB | ✓ | 1 | check ↗ |
| gemma 7b | llm | 31.95tokens/s | 8GB | ✓ | 1 | check ↗ |
| falcon2 11b | llm | 31.2tokens/s | 8GB | ✓ | 1 | check ↗ |
| Gemma 2 9B | llm | 23.8tokens/s | 8GB | ✓ | 1 | check ↗ |
| stablelm2 12b | llm | 18.73tokens/s | 8GB | ✓ | 1 | check ↗ |
| llama2 13b | llm | 9.25tokens/s | 8GB | ✓ | 1 | check ↗ |