§01·model · /models
Llama 3.1 8B
llmactiveLlama 3.1 Community License
8B instruction-tuned LLM by Meta (Llama 3.1) with 128K context - the workhorse small Llama. Llama 3.1 Community License.
§02·GPUs that run this model
23 total| GPU | VRAM | Series | Best speed | Min VRAM | Works | Benchmarks | Recipe | |
|---|---|---|---|---|---|---|---|---|
| RTX 4070 Ti | 12GB | 40 | 60tokens/s | ✓ | 2 | recipe | check ↗ | |
| RTX 3060 Ti | 8GB | 30 | 57.34tokens/s | 8GB | ✓ | 1 | recipe | check ↗ |
| RTX 3060 | 12GB | 30 | 52.2tokens/s | ✓ | 2 | recipe | check ↗ | |
| RX 7900 XTX | 24GB | amd | 51.3tokens/s | 24GB | ✓ | 1 | recipe | check ↗ |
| Apple M2 Max | 64GB | apple | ~ | 0 | recipe | check ↗ | ||
| Apple M2 Pro | 16GB | apple | ~ | 0 | recipe | check ↗ | ||
| Apple M3 Max | 48GB | apple | ~ | 0 | recipe | check ↗ | ||
| Apple M4 Max | 48GB | apple | ~ | 0 | recipe | check ↗ | ||
| RTX 3080 Ti | 12GB | 30 | ~ | 0 | recipe | check ↗ | ||
| RTX 3090 | 24GB | 30 | ~ | 0 | recipe | check ↗ | ||
| RTX 3090 Ti | 24GB | 30 | ~ | 0 | recipe | check ↗ | ||
| RTX 4070 | 12GB | 40 | ~ | 0 | recipe | check ↗ | ||
| RTX 4070 Super | 12GB | 40 | ~ | 0 | recipe | check ↗ | ||
| RTX 4070 Ti Super | 16GB | 40 | ~ | 0 | recipe | check ↗ | ||
| RTX 4080 | 16GB | 40 | ~ | 0 | recipe | check ↗ | ||
| RTX 4080 Super | 16GB | 40 | ~ | 0 | recipe | check ↗ | ||
| RTX 4090 | 24GB | 40 | ~ | 0 | recipe | check ↗ | ||
| RTX 5060 Ti | 16GB | 50 | ~ | 0 | recipe | check ↗ | ||
| RTX 5070 | 12GB | 50 | ~ | 0 | recipe | check ↗ | ||
| RTX 5070 Ti | 16GB | 50 | ~ | 0 | recipe | check ↗ | ||
| RTX 5080 | 16GB | 50 | ~ | 0 | recipe | check ↗ | ||
| RTX 5090 | 32GB | 50 | ~ | 0 | recipe | check ↗ | ||
| RX 7800 XT | 16GB | amd | ~ | 0 | recipe | check ↗ |
✓ benchmarked·~ runs via recipe (not benchmarked)·— untested·✕doesn't fit