Community performance index
Real hardware.
Real numbers.
A living map of what local models can do on AMD-powered Macs. Standardized runs, honest context, and community data you can reproduce.
Submit your benchmark ↗Runs available358and growing
Models tested79Dense + MoE
GPU families51RDNA + GCN
Fastest generation297.01tokens / sec
| Model / quant | Hardware | Architecture | Prompt | Generation | Source |
|---|---|---|---|---|---|
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 8088.69tok/s | 297.01tok/s | Dolphins1972Jul 23, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 5655.79tok/s | 288.64tok/s | BirdsLikeWiresJul 22, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 5476.18tok/s | 268.45tok/s | Dolphins1972Jul 18, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 8153.51tok/s | 266.14tok/s | Dolphins1972Jul 23, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 8077.28tok/s | 265.52tok/s | Dolphins1972Jul 23, 2026 |
Llama 3.2 1BQ4_K_M · Dense · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 2994tok/s | 250tok/s | ToshLLM LabJul 15, 2026 |
Llama 3.2 1BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 5805.79tok/s | 233.02tok/s | BirdsLikeWiresJul 22, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 6491.86tok/s | 207.43tok/s | ToshLLM communitySep 9, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 6224.67tok/s | 206.18tok/s | ToshLLM communityAug 10, 2026 |
Nomic Embed Text V2 MoEQ8_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 2195.09tok/s | 179.48tok/s | mitchdeSep 29, 2026 |
Nomic Embed Text V2 MoEQ8_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 1378.77tok/s | 167.97tok/s | mitchdeSep 11, 2026 |
Nomic Embed Text V2 MoEQ8_0 · MoE · Signed ToshLLM app · ncmoe 1 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 2188.48tok/s | 165.08tok/s | mitchdeSep 25, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 2289.14tok/s | 163.8tok/s | ToshLLM communitySep 22, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 2220.1tok/s | 161.48tok/s | ToshLLM communitySep 22, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 5211.36tok/s | 157.49tok/s | Dolphins1972Jul 23, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 4591.79tok/s | 151.79tok/s | PixlaAug 29, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 4478.11tok/s | 142.67tok/s | Dolphins1972Jul 18, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 6342.45tok/s | 129.18tok/s | ToshLLM communityAug 21, 2026 |
MiniCPM5 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 2058.84tok/s | 125.07tok/s | mnerdSep 30, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 1652.01tok/s | 118.87tok/s | Dolphins1972Jul 23, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo31.98 GB VRAM | GCN / Vega | 2269.3tok/s | 114.01tok/s | chafeyAug 27, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon Vega31.97 GB VRAM | RDNA 2 + GCN / Vega | 3047.25tok/s | 113.04tok/s | ToshLLM communityAug 10, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 1473.96tok/s | 112.92tok/s | mnerdSep 7, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 3730.57tok/s | 110.75tok/s | ToshLLM communityJul 31, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 1334.01tok/s | 110.45tok/s | gervaiscoquil✓Jul 27, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 1623.43tok/s | 106.16tok/s | Dolphins1972Jul 18, 2026 |
LLaMA V2 7BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1508.5tok/s | 105.77tok/s | ToshLLM communityAug 6, 2026 |
MiniCPM5 1BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 2089.32tok/s | 105.13tok/s | ToshLLM communityAug 4, 2026 |
Llama 3.2 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 1602.61tok/s | 103.89tok/s | mnerdAug 31, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 1293.29tok/s | 99.19tok/s | ToshLLM LabSep 1, 2026 |
Gemma 4 19BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1881.52tok/s | 99.11tok/s | ToshLLM communityAug 6, 2026 |
Qwen3 4BQ4_K_M · Dense · comparison | RX 6900 XT16 GB VRAM | RDNA 2 | 291.4tok/s | 97.5tok/s | Community testerJul 16, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 2275.38tok/s | 96.88tok/s | simmc303Aug 26, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 1083.66tok/s | 94.99tok/s | ToshLLM LabJul 27, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 1293.17tok/s | 94.98tok/s | gervaiscoquil✓Jul 22, 2026 |
GLM 4.7 Flash 23B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 1 · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1341.71tok/s | 94.83tok/s | ToshLLM communityAug 6, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 1038.29tok/s | 94.59tok/s | ToshLLM LabJul 24, 2026 |
Gemma 4 A4B 98e V6 Coder ItQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1773.77tok/s | 94.17tok/s | ToshLLM communityAug 6, 2026 |
Qwen3 4BQ4_K_M · Dense · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 1005tok/s | 94tok/s | ToshLLM LabJul 15, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 1516.84tok/s | 93.53tok/s | ToshLLM communitySep 3, 2026 |
Liquid A1B 8BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 896.26tok/s | 93.22tok/s | mnerdSep 7, 2026 |
gpt-oss 20BQ4_K_M · MoE · Full VRAM · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 972tok/s | 93tok/s | ToshLLM LabJul 15, 2026 |
GPT OSS 20BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 1037.06tok/s | 92.94tok/s | ToshLLM LabJul 22, 2026 |
Gemma 3 4BQ4_K_M · Dense · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 1168tok/s | 90tok/s | ToshLLM LabJul 15, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 1271.9tok/s | 89.97tok/s | Flint Ironstag✓Jul 18, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 1236.49tok/s | 89.8tok/s | ToshLLM communitySep 22, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 1971.4tok/s | 89.79tok/s | ardentwheelJul 20, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 1146.18tok/s | 87.63tok/s | ToshLLM LabJul 17, 2026 |
Llama 3.2 1BUD-Q5_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 1388.43tok/s | 85.26tok/s | mnerdJul 31, 2026 |
Liquid LFM2 8BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 791.04tok/s | 85.01tok/s | mnerdAug 29, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 2143.87tok/s | 81.82tok/s | ToshLLM communityAug 10, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 828.51tok/s | 77.02tok/s | Dolphins1972Jul 18, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500 XT7.98 GB VRAM | RDNA 1 | 1300.69tok/s | 76.92tok/s | ToshLLM communityOct 9, 2026 |
Feb5E04Da4910Dd56D33F0Cd03747Ff298C1E801 8B-A1BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 787.47tok/s | 76.28tok/s | mnerdAug 29, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 945.19tok/s | 75.93tok/s | ToshLLM communityOct 2, 2026 |
Ornith 1.5 35BUD-IQ3_XXS · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1602.21tok/s | 75.5tok/s | Quad849Sep 4, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 66007.98 GB VRAM | RDNA 2 | 2258.62tok/s | 74.17tok/s | ToshLLM communitySep 26, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 1318.81tok/s | 73.54tok/s | gervaiscoquil✓Jul 22, 2026 |
GPT OSS 20BMXFP4 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X15.98 GB VRAM | RDNA 1 | 705.92tok/s | 73.3tok/s | ToshLLM communityJul 18, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 1318.13tok/s | 73.29tok/s | ToshLLM community✓Jul 22, 2026 |
LLaMA V2 7BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 655.45tok/s | 72.73tok/s | ToshLLM LabJul 20, 2026 |
Models Zai Org GLM 4.6V FlashQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1131.82tok/s | 72.12tok/s | ToshLLM communityAug 6, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 1046.58tok/s | 71.54tok/s | ToshLLM communitySep 11, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 1253.18tok/s | 71.02tok/s | chafeyAug 25, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 1262.58tok/s | 70.86tok/s | PixlaAug 29, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 1254.93tok/s | 70.69tok/s | chafeyAug 23, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 1199.09tok/s | 69.2tok/s | simmc303Aug 26, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 787.35tok/s | 67.34tok/s | warriordr7Oct 8, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 612.88tok/s | 67tok/s | Flint Ironstag✓Jul 18, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 611.58tok/s | 66.87tok/s | ToshLLM community✓Jul 18, 2026 |
Ornith 1.5 35BUD-IQ3_XXS · MoE · Signed ToshLLM app · ncmoe 1 · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 1105.37tok/s | 66.28tok/s | Quad849Sep 4, 2026 |
Openai GPT OSS 120BMXFP4 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 266.48tok/s | 65.99tok/s | threadedSep 14, 2026 |
Gemma 4 26B-A4BQ8_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 765.81tok/s | 65.41tok/s | warriordr7Oct 8, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 601.17tok/s | 65.24tok/s | Flint Ironstag✓Jul 18, 2026 |
Qwen2.5 Coder 1.5BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 640.27tok/s | 65.23tok/s | cloudphreniaSep 10, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 1425.95tok/s | 64.29tok/s | threadedSep 16, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 1082.36tok/s | 63.55tok/s | BirdsLikeWiresJul 22, 2026 |
GPT OSS 20BMXFP4 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega 6415.98 GB VRAM | GCN / Vega | 572.65tok/s | 63.51tok/s | goroSep 24, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 1143.21tok/s | 63.38tok/s | PixlaAug 2, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X31.98 GB VRAM | RDNA 2 | 1156.32tok/s | 63.26tok/s | XanthousJul 28, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 1139.23tok/s | 63.14tok/s | Flint Ironstag✓Aug 22, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 1171.32tok/s | 62.94tok/s | Flint Ironstag✓Aug 22, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 899.15tok/s | 62.76tok/s | ArdentwheelJul 20, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 1179.97tok/s | 60.54tok/s | めろんSep 7, 2026 |
Qwen3 8BQ4_K_M · Dense · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 476tok/s | 60tok/s | ToshLLM LabJul 15, 2026 |
Ornith 1.5 35BQ5_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X ×231.97 GB VRAM | RDNA 1 | 791.23tok/s | 59.61tok/s | HypabolicOct 12, 2026 |
Hf Format 1.7BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 547.13tok/s | 58.7tok/s | cloudphreniaSep 22, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 963.76tok/s | 57.57tok/s | mnerdJul 31, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 904.07tok/s | 57.49tok/s | threadedSep 16, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 945.34tok/s | 57.36tok/s | threadedSep 16, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 948.35tok/s | 57.33tok/s | threadedSep 16, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 905.89tok/s | 57.2tok/s | threadedSep 16, 2026 |
Hf Format 1.7BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 583.94tok/s | 56.89tok/s | cloudphreniaSep 22, 2026 |
Qwen3.6 14B A3BQ5_K_M · MoE · Full VRAM · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 736tok/s | 56tok/s | ToshLLM LabJul 15, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 904.81tok/s | 55.83tok/s | threadedSep 15, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 1086.98tok/s | 55.79tok/s | PixlaAug 2, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 1102.7tok/s | 55.75tok/s | Flint Ironstag✓Aug 11, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 587.58tok/s | 55.42tok/s | ToshLLM communitySep 3, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 528.49tok/s | 55.39tok/s | Flint Ironstag✓Jul 18, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 687.55tok/s | 55.06tok/s | ToshLLM communitySep 3, 2026 |
Gemma 4 E4BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon VII15.98 GB VRAM | GCN / Vega | 825.1tok/s | 54.97tok/s | 1252er✓Jul 21, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 617.65tok/s | 54.89tok/s | simmc303Aug 28, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 952.42tok/s | 54.76tok/s | Flint Ironstag✓Jul 18, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 940.75tok/s | 54.54tok/s | Flint Ironstag✓Jul 18, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 576.46tok/s | 54.07tok/s | simmc303Sep 23, 2026 |
Gemma 4 12BQ4_K_M · Dense · comparison | RX 6950 XT16 GB VRAM | RDNA 2 | 595.7tok/s | 53.4tok/s | Community testerJul 16, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 346.78tok/s | 53.32tok/s | gervaiscoquil✓Jul 18, 2026 |
Qwen3.6 12BQ5_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 855.26tok/s | 52.83tok/s | ToshLLM communityAug 6, 2026 |
Qwen2.5 Coder 7BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5700 XT7.98 GB VRAM | RDNA 1 | 495.6tok/s | 52.31tok/s | blacklizardJul 31, 2026 |
Qwen3 4BQ4_K_M · Dense · comparison | RX 5600 XT6 GB VRAM | RDNA 1 | 100tok/s | 52.1tok/s | Community testerJul 16, 2026 |
Qwen3.8 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 756.54tok/s | 52.03tok/s | ToshLLM LabAug 21, 2026 |
LFM2.5 VL 3BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 352.07tok/s | 51.65tok/s | cloudphreniaSep 14, 2026 |
MiniCPM5 2.6BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 775.26tok/s | 51.53tok/s | ToshLLM communitySep 16, 2026 |
GPT OSS 20BMXFP4 · MoE · Signed ToshLLM app · ncmoe 3 · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 653.17tok/s | 51.35tok/s | ToshLLM communityOct 2, 2026 |
Gemma 4 E4BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 558.48tok/s | 51.25tok/s | ToshLLM communitySep 8, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Vega 567.98 GB VRAM | GCN / Vega | 743.66tok/s | 51.09tok/s | ToshLLM communitySep 6, 2026 |
Qwen K-quantK-quant · Dense · Wave64 path; model details pending verification · comparison | RX 5808 GB VRAM | GCN / Vega | 285tok/s | 51tok/s | Community testerJul 11, 2026 |
Gemma4 26B-A4BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 1758.66tok/s | 50.89tok/s | ToshLLM communitySep 6, 2026 |
Gemma4 V2 Merged 12BFP16 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 833tok/s | 50.83tok/s | ToshLLM communityAug 6, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 800.89tok/s | 49.62tok/s | めろんOct 7, 2026 |
Ornith 1.0 35BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X ×231.97 GB VRAM | RDNA 1 | 703.09tok/s | 49.49tok/s | HypabolicAug 12, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 552.59tok/s | 48.8tok/s | ToshLLM community✓Jul 20, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 436.19tok/s | 48.71tok/s | めろんSep 4, 2026 |
Nemotron 3 Nano Omni 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | Apple M1 Max51.84 GB VRAM | Apple Silicon | 541.73tok/s | 48.62tok/s | simmc303Sep 22, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 757.14tok/s | 48.49tok/s | Dolphins1972Jul 23, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 825.73tok/s | 48.46tok/s | ToshLLM communityJul 24, 2026 |
Qwen3.5 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 650.88tok/s | 48.38tok/s | ToshLLM LabJul 27, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 484.84tok/s | 48.18tok/s | gervaiscoquil✓Jul 18, 2026 |
Gemma 4 26B A4BQ4_K_M · MoE · comparison | Radeon Pro Vega II32 GB VRAM | GCN / Vega | 190.3tok/s | 47.9tok/s | Mac Pro communityJul 16, 2026 |
Qwen3-Coder 30B A3BQ4_K_M · MoE · ncmoe 23 · comparison | RX 6950 XT16 GB VRAM | RDNA 2 | 587.3tok/s | 47.8tok/s | Community testerJul 16, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 483.53tok/s | 47.63tok/s | ButterChickenJul 20, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X15.98 GB VRAM | RDNA 1 | 528.85tok/s | 47.54tok/s | alifeinbinaryOct 5, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 488.41tok/s | 47.44tok/s | ToshLLM communityOct 2, 2026 |
MiniCPM5 2.6BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 395.82tok/s | 47.27tok/s | cloudphreniaSep 10, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 800.04tok/s | 47.1tok/s | ToshLLM communityAug 1, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT ×231.97 GB VRAM | RDNA 2 | 585.78tok/s | 46.89tok/s | Dolphins1972Jul 18, 2026 |
Qwen3 14BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 552.39tok/s | 46.56tok/s | chafeyAug 23, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X ×231.97 GB VRAM | RDNA 1 | 704.73tok/s | 46.53tok/s | HypabolicAug 12, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 436.77tok/s | 46.46tok/s | Aaron.CAug 11, 2026 |
Qwen3.5 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 414.37tok/s | 46.46tok/s | ToshLLM LabJul 17, 2026 |
Gemma4 V2 MergedQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 799.02tok/s | 45.89tok/s | ToshLLM communityAug 1, 2026 |
UncensoredLM DeepSeek R1 Qwen 13.5BQ5_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB15.98 GB VRAM | RDNA | 720.61tok/s | 45.53tok/s | ToshLLM communityAug 6, 2026 |
Qwen3.5 9BQ4_K_M · Dense · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 411tok/s | 45tok/s | ToshLLM LabJul 15, 2026 |
LFM2.5 8B-A1BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 4 · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 374.39tok/s | 44.37tok/s | cloudphreniaSep 14, 2026 |
GPT OSS 20BF16 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon Vega31.97 GB VRAM | RDNA 2 + GCN / Vega | 802.94tok/s | 43.92tok/s | ToshLLM communityAug 11, 2026 |
Nanbeige4.1 3BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 292.99tok/s | 43.63tok/s | cloudphreniaSep 14, 2026 |
Smollm3 3BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 315.9tok/s | 43.32tok/s | cloudphreniaSep 14, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | Apple M1 Max51.84 GB VRAM | Apple Silicon | 645.73tok/s | 43.04tok/s | simmc303Sep 22, 2026 |
Ornith 1.5 35BBF16 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 519.13tok/s | 42.78tok/s | threadedSep 14, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon Vega31.97 GB VRAM | RDNA 2 + GCN / Vega | 1216.76tok/s | 42.52tok/s | ToshLLM communityAug 10, 2026 |
Qwen3 4BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 236.75tok/s | 42.43tok/s | cloudphreniaSep 22, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 280.34tok/s | 42.24tok/s | gervaiscoquil✓Jul 18, 2026 |
Qwen3.6 35B A3BQ4_K_S · MoE · All experts in VRAM · comparison | Radeon Pro Vega II32 GB VRAM | GCN / Vega | 284.7tok/s | 42.1tok/s | Mac Pro communityJul 16, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 570.26tok/s | 42.02tok/s | ardentwheelAug 1, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 432.42tok/s | 41.97tok/s | ToshLLM communityAug 4, 2026 |
Qwen3.6 35B-A3BUD-Q8_K_XL · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 320.65tok/s | 41.94tok/s | ToshLLM communityAug 5, 2026 |
Qwen3.5 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5700 XT7.98 GB VRAM | RDNA 1 | 432.01tok/s | 41.46tok/s | blacklizardJul 31, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×4 + AMD Radeon PRO W6800X Duo ×2191.88 GB VRAM | GCN / Vega + RDNA 2 | 794.46tok/s | 41.07tok/s | Flint Ironstag✓Jul 20, 2026 |
Hf E4BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Vega 567.98 GB VRAM | GCN / Vega | 644.1tok/s | 40.54tok/s | めろんAug 25, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 429.66tok/s | 40.3tok/s | Flint Ironstag✓Jul 18, 2026 |
Google Gemma 4 E2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 394.48tok/s | 40.28tok/s | cloudphreniaSep 22, 2026 |
Llama 3.2 1BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 218.84tok/s | 40.21tok/s | ToshLLM communitySep 23, 2026 |
Llama 3.2 1BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 219.08tok/s | 40.06tok/s | toshllm@wbig.gsSep 22, 2026 |
Nanbeige4.1 3BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 276.21tok/s | 39.57tok/s | cloudphreniaSep 14, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5300M3.98 GB VRAM | RDNA 1 | 453.2tok/s | 39.36tok/s | laukaaOct 12, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 384.95tok/s | 38.71tok/s | Aaron.CAug 11, 2026 |
Gemma 3 1BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 319.54tok/s | 38.48tok/s | toshllm@wbig.gsSep 22, 2026 |
Gemma 4 E2BQ4_K · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 421.41tok/s | 38.44tok/s | cloudphreniaSep 10, 2026 |
Phi 4 Mini InstructQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 264.77tok/s | 38.43tok/s | cloudphreniaSep 14, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 271.35tok/s | 37.72tok/s | htweedieSep 5, 2026 |
Hf Format 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 229.83tok/s | 36.78tok/s | cloudphreniaSep 22, 2026 |
Gemma 4 E4BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 394.92tok/s | 36.36tok/s | mnerdSep 30, 2026 |
Bonsai 8BQ1_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 133.93tok/s | 36.05tok/s | cloudphreniaSep 14, 2026 |
Qwen3 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 157.92tok/s | 35.84tok/s | ToshLLM communityAug 21, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | Apple M1 Max51.84 GB VRAM | Apple Silicon | 340.89tok/s | 35.68tok/s | simmc303Sep 22, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 368.27tok/s | 35.05tok/s | めろんSep 4, 2026 |
Gemma 4 E4BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 334.86tok/s | 34.86tok/s | mnerdJul 30, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 337.18tok/s | 34.66tok/s | ToshLLM LabJul 18, 2026 |
Llama 3.2 3BUD-Q6_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 473.51tok/s | 34.65tok/s | mnerdJul 31, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 277.72tok/s | 34.4tok/s | ToshLLM community✓Jul 20, 2026 |
Granite 4.2 3BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 232.69tok/s | 34.13tok/s | cloudphreniaSep 14, 2026 |
Gemma 4 12BQ4_K_XL · Dense · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 342tok/s | 34tok/s | ToshLLM LabJul 15, 2026 |
Qwen2.5 1.5BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 255.5tok/s | 33.67tok/s | mitchdeSep 12, 2026 |
Hf 27BPQ2_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 255.55tok/s | 33.5tok/s | ToshLLM LabSep 23, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 10 · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 194.85tok/s | 33.5tok/s | tmnJul 18, 2026 |
Qwen3 4BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 232.34tok/s | 33.35tok/s | cloudphreniaSep 22, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 266.06tok/s | 33.27tok/s | ToshLLM community✓Jul 20, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon Vega31.97 GB VRAM | RDNA 2 + GCN / Vega | 807.25tok/s | 33.2tok/s | ToshLLM communityAug 10, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×6 + AMD Radeon Pro 580X103.91 GB VRAM | GCN / Vega | 558.18tok/s | 32.86tok/s | Flint IronstagSep 10, 2026 |
GLM 4.7 Flash 23B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon GFX10, 16 GB + AMD Radeon RX 5600 XT21.97 GB VRAM | RDNA + RDNA 1 | 925.86tok/s | 32.66tok/s | ToshLLM communityAug 7, 2026 |
Hf Format 4BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 220.64tok/s | 32.6tok/s | cloudphreniaSep 22, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 315.13tok/s | 31.78tok/s | ToshLLM communitySep 3, 2026 |
Qwen2.5 Coder 1.5BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 266.32tok/s | 31.63tok/s | mitchdeSep 12, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×6 + AMD Radeon Pro 580X103.91 GB VRAM | GCN / Vega | 548.52tok/s | 31.45tok/s | Flint IronstagSep 10, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 252.37tok/s | 31.39tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×4 + AMD Radeon PRO W6800X Duo ×2191.88 GB VRAM | GCN / Vega + RDNA 2 | 426.35tok/s | 31.28tok/s | Flint Ironstag✓Jul 20, 2026 |
Gemma 4 E4BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 214.19tok/s | 31.19tok/s | cloudphreniaSep 22, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT15.98 GB VRAM | RDNA 2 | 763.1tok/s | 30.98tok/s | ToshLLM communityAug 10, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 4804 GB VRAM | GCN / Vega | 148.35tok/s | 30.89tok/s | ToshLLM communityOct 11, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×4 + AMD Radeon PRO W6800X Duo ×2191.88 GB VRAM | GCN / Vega + RDNA 2 | 408.46tok/s | 30.83tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.5 4BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 186.87tok/s | 30.79tok/s | cloudphreniaSep 22, 2026 |
Gemma4 26B-A4BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon Vega31.97 GB VRAM | RDNA 2 + GCN / Vega | 883.21tok/s | 30.44tok/s | ToshLLM communityAug 21, 2026 |
Qwen2.5 Coder 7BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 138.96tok/s | 30.07tok/s | cloudphreniaSep 10, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 311.61tok/s | 30.01tok/s | ToshLLM communitySep 11, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · ncmoe 2 · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 393.64tok/s | 29.91tok/s | めろんSep 4, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 313.59tok/s | 29.46tok/s | ToshLLM communityOct 2, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×4 + AMD Radeon PRO W6800X Duo ×2191.88 GB VRAM | GCN / Vega + RDNA 2 | 396.42tok/s | 29.41tok/s | Flint Ironstag✓Jul 20, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro WX 5100 + AMD Radeon Vega23.98 GB VRAM | GCN / Vega | 199.04tok/s | 29.26tok/s | めろんOct 9, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 17 · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 341.07tok/s | 29.09tok/s | tmnJul 18, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 348.2tok/s | 29.06tok/s | tmnJul 18, 2026 |
Smollm3 3BBF16 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 383.9tok/s | 28.55tok/s | cloudphreniaSep 14, 2026 |
Qwen3 VL 2BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 147.37tok/s | 28.26tok/s | ToshLLM communitySep 23, 2026 |
Qwen3.8 Flash NextUD-Q4_K_XL · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon PRO W6800X Duo ×2143.92 GB VRAM | RDNA 2 | 490.19tok/s | 28.03tok/s | chafeySep 13, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro WX 5100 + AMD Radeon Vega23.98 GB VRAM | GCN / Vega | 138.14tok/s | 27.87tok/s | めろんSep 3, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega 203.98 GB VRAM | GCN / Vega | 221.16tok/s | 27.64tok/s | uwe_aSep 13, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 43 · amd-gpu · Metal · comparison | AMD Radeon RX 5600 XT5.98 GB VRAM | RDNA 1 | 138.37tok/s | 27.52tok/s | mitchdeOct 4, 2026 |
Qwen3 VL 2BQ3_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 229.37tok/s | 27.52tok/s | mitchdeSep 11, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 368.16tok/s | 27.49tok/s | Aaron.CAug 11, 2026 |
Gemma4 V2 MergedQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6900 XT + AMD Radeon Pro W5700X31.97 GB VRAM | RDNA 2 + RDNA 1 | 466.28tok/s | 27.44tok/s | ToshLLM communityAug 1, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 24 · amd-gpu · Metal · comparison | AMD Radeon RX 6700 XT11.98 GB VRAM | RDNA 2 | 473.74tok/s | 27.34tok/s | ToshLLM LabSep 11, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×231.97 GB VRAM | GCN / Vega | 793.59tok/s | 27.24tok/s | ToshLLM communityJul 23, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×4 + AMD Radeon PRO W6800X Duo ×2191.88 GB VRAM | GCN / Vega + RDNA 2 | 373.62tok/s | 27.19tok/s | Flint Ironstag✓Jul 20, 2026 |
Ornith 1.5 9BBF16 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 432.19tok/s | 27.18tok/s | threadedSep 14, 2026 |
Gemma 3 1BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon HD - FirePro D300 ×24 GB VRAM | GCN / Vega | 301.39tok/s | 26.54tok/s | めろんOct 1, 2026 |
Qwen3.8 27BUD-IQ3_XXS · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 6800 XT15.98 GB VRAM | RDNA 2 | 338.01tok/s | 26.43tok/s | ToshLLM communitySep 2, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | Apple M1 Max51.84 GB VRAM | Apple Silicon | 248.83tok/s | 26.31tok/s | simmc303Sep 22, 2026 |
Qwen3.5 122B A10BQ5_K_M · MoE · Four-GPU split · comparison | 4× W6800X128 GB VRAM | RDNA 2 | 226.5tok/s | 26.1tok/s | Basten7Jul 15, 2026 |
Gemma 4 E4BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 206.88tok/s | 25.64tok/s | cloudphreniaSep 22, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 222.53tok/s | 25.36tok/s | ToshLLM communitySep 8, 2026 |
Gemma 4 E4BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 216.37tok/s | 25.27tok/s | 1252er✓Aug 2, 2026 |
Gemma 4 E4BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 210.8tok/s | 25.12tok/s | 1252er✓Jul 21, 2026 |
Qwen3.6 35B A3BQ4_K_S · MoE · CPU expert offload · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 434tok/s | 25tok/s | ToshLLM LabJul 15, 2026 |
Granite 4.2 3BBF16 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 259.3tok/s | 24.26tok/s | cloudphreniaSep 14, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Vega 567.98 GB VRAM | GCN / Vega | 258.72tok/s | 24.24tok/s | めろんAug 25, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×231.97 GB VRAM | GCN / Vega | 427.04tok/s | 23.94tok/s | ToshLLM communityJul 23, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · ncmoe 4 · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 351.94tok/s | 23.83tok/s | めろんSep 11, 2026 |
Ornith 1.5 35BBF16 · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II + AMD Radeon PRO W6800X Duo95.95 GB VRAM | GCN / Vega + RDNA 2 | 665.15tok/s | 23.72tok/s | threadedSep 15, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 532.51tok/s | 23.49tok/s | ToshLLM community✓Jul 20, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 532.87tok/s | 23.48tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.8 27BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo31.98 GB VRAM | RDNA 2 | 280.26tok/s | 23.45tok/s | PixlaAug 29, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 229.68tok/s | 22.98tok/s | ToshLLM communityAug 30, 2026 |
Mistralai Mistral 7BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 122.26tok/s | 22.62tok/s | cloudphreniaSep 14, 2026 |
Ornith 1.5 9BQ6_K · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 105.04tok/s | 22.5tok/s | cloudphreniaSep 14, 2026 |
V2.1 27B-A18B 27B-A18BQ4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 224.55tok/s | 22.23tok/s | ToshLLM communitySep 22, 2026 |
Gemma 4 26B A4BMXFP4 · MoE · CPU expert offload · comparison | RX 6700 XT12 GB VRAM | RDNA 2 | 552tok/s | 22tok/s | ToshLLM LabJul 15, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon Pro 5700 XT15.98 GB VRAM | RDNA 1 | 424.08tok/s | 21.9tok/s | Aaron.CAug 11, 2026 |
Qwen2.5 3BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 127.77tok/s | 21.04tok/s | ToshLLM communitySep 1, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 232.37tok/s | 20.94tok/s | ToshLLM community✓Jul 20, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega15.98 GB VRAM | GCN / Vega | 230.23tok/s | 20.89tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 237.43tok/s | 20.22tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.8 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 167.97tok/s | 20.18tok/s | mnerdAug 26, 2026 |
Gemma 4 26B-A4BUD-Q4_K_XL · MoE · Signed ToshLLM app · ncmoe 3 · amd-gpu · Metal · comparison | AMD Radeon RX Vega 56 + AMD Radeon Pro WX 510015.98 GB VRAM | GCN / Vega | 108.98tok/s | 20.1tok/s | めろんAug 25, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · ncmoe 2 · amd-gpu · Metal · comparison | AMD Radeon RX Vega 56 + AMD Radeon Pro WX 510015.98 GB VRAM | GCN / Vega | 111.45tok/s | 20.09tok/s | めろんAug 25, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 167.83tok/s | 20.06tok/s | test1-Henbury-DIsableJul 20, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · ncmoe 2 · amd-gpu · Metal · comparison | AMD Radeon RX Vega 56 + AMD Radeon Pro WX 510015.98 GB VRAM | GCN / Vega | 111.01tok/s | 19.99tok/s | めろんAug 25, 2026 |
Qwen3Vl 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5808 GB VRAM | GCN / Vega | 239.43tok/s | 19.86tok/s | ToshLLM communityAug 3, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 164.3tok/s | 19.86tok/s | ardentwheelJul 19, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 166.97tok/s | 19.51tok/s | test0-Henbury-DIsableJul 20, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 1438.6tok/s | 19.48tok/s | PixlaAug 29, 2026 |
GLM 4.5 AirQ4_K_M · MoE · Signed ToshLLM app · ncmoe 11 · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 199.11tok/s | 19.4tok/s | warriordr7Oct 8, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×4 + AMD Radeon PRO W6800X Duo ×2191.88 GB VRAM | GCN / Vega + RDNA 2 | 328.49tok/s | 19.1tok/s | Flint Ironstag✓Jul 20, 2026 |
Unsloth Gguf 4Yqurthx 2BQ6_K · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 147.87tok/s | 18.52tok/s | ToshLLM communitySep 23, 2026 |
Hf 26BQ4_0 · MoE · Signed ToshLLM app · ncmoe 3 · amd-gpu · Metal · comparison | AMD Radeon RX Vega 56 + AMD Radeon Pro WX 510015.98 GB VRAM | GCN / Vega | 126.13tok/s | 18.23tok/s | めろんAug 12, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 271.81tok/s | 17.97tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.8 27BUD-Q8_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 164.6tok/s | 17.79tok/s | warriordr7Oct 8, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 262.11tok/s | 17.59tok/s | Flint Ironstag✓Jul 20, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 22 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 231.85tok/s | 17.29tok/s | ArdentwheelJul 20, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 247.23tok/s | 17.15tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 170.51tok/s | 17.11tok/s | ToshLLM communitySep 3, 2026 |
Qwen3.8 27BUD-Q4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II31.98 GB VRAM | GCN / Vega | 143.86tok/s | 17.11tok/s | simmc303Aug 25, 2026 |
Qwen3.6 35B-A3BUD-Q2_K_XL · MoE · Signed ToshLLM app · ncmoe 23 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 287.01tok/s | 16.88tok/s | ToshLLM communitySep 7, 2026 |
DeepSeek Coder V2 Lite InstructIQ2_XS · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 111.72tok/s | 16.37tok/s | mitchdeSep 24, 2026 |
Qwen3.8 27BUD-IQ3_S · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X15.98 GB VRAM | RDNA 1 | 148.95tok/s | 16.16tok/s | alifeinbinaryOct 1, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 35 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 141.93tok/s | 15.79tok/s | ToshLLM communitySep 10, 2026 |
Gemma 4 31BQ8_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 156.34tok/s | 15.78tok/s | warriordr7Oct 8, 2026 |
Gemma4 26B-A4BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 20 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 152.43tok/s | 14.95tok/s | ToshLLM communitySep 8, 2026 |
Google Gemma 4 31BQ4_0 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | Apple M1 Max51.84 GB VRAM | Apple Silicon | 101.03tok/s | 14.91tok/s | simmc303Sep 22, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×231.97 GB VRAM | GCN / Vega | 319.02tok/s | 14.86tok/s | ToshLLM communityJul 23, 2026 |
Qwen2.5 Coder 7BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5808 GB VRAM | GCN / Vega | 144.55tok/s | 14.75tok/s | ToshLLM communityAug 3, 2026 |
Carnice Qwen3.6 MoE 35BQ3_K_M · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 99.63tok/s | 14.52tok/s | 1252er✓Aug 2, 2026 |
Qwen3.6 27BIQ4 · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro W5700X ×231.97 GB VRAM | RDNA 1 | 127.25tok/s | 14.25tok/s | malzzzJul 23, 2026 |
GPT OSS 20BMXFP4 · MoE · Signed ToshLLM app · ncmoe 15 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 188.97tok/s | 14.22tok/s | mnerdJul 31, 2026 |
Qwen3.8 27BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo ×2127.94 GB VRAM | GCN / Vega | 142.67tok/s | 14.11tok/s | ToshLLM communityAug 24, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 164.7tok/s | 14.07tok/s | ardentwheelJul 18, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 164.44tok/s | 14.03tok/s | test1-Henbury-EnableJul 20, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon RX 6600 XT7.98 GB VRAM | RDNA 2 | 164.13tok/s | 13.97tok/s | test0-Henbury-EnableJul 20, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 35 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 141.58tok/s | 13.96tok/s | ToshLLM communitySep 10, 2026 |
DeepSeek Coder V2 Lite InstructIQ2_XS · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 111.55tok/s | 13.52tok/s | mitchdeSep 25, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Vega 56 + AMD Radeon Pro WX 510015.98 GB VRAM | GCN / Vega | 58.75tok/s | 13.41tok/s | めろんAug 25, 2026 |
Gemma 4 12BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Vega ×463.94 GB VRAM | GCN / Vega | 228.36tok/s | 13.3tok/s | Flint Ironstag✓Jul 20, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 152.03tok/s | 13.22tok/s | oldwongSep 22, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 148.47tok/s | 13.21tok/s | ToshLLM communitySep 5, 2026 |
GLM 4 9BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 106.9tok/s | 13.2tok/s | 1252er✓Aug 2, 2026 |
Ornith 1.5 35BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 138.8tok/s | 13.19tok/s | ToshLLM communitySep 10, 2026 |
Ornith 1.5 35B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 142.36tok/s | 13.13tok/s | oldwongSep 22, 2026 |
Qwen3.8 27BUD-IQ2_S · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega 6415.98 GB VRAM | GCN / Vega | 104.51tok/s | 13.11tok/s | goroSep 24, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 148.37tok/s | 13.09tok/s | oldwongSep 16, 2026 |
Ornith 1.5 35B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 138.68tok/s | 13.06tok/s | oldwongSep 16, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 30 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 149.38tok/s | 13.05tok/s | ToshLLM communitySep 5, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 141.19tok/s | 12.86tok/s | ToshLLM communitySep 4, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 175.87tok/s | 12.82tok/s | ToshLLM communitySep 3, 2026 |
LLaMA V2 7BQ4_0 · Dense · Signed ToshLLM app · standard-auto · Metal · comparison | Apple M111.84 GB VRAM | Apple Silicon | 128.09tok/s | 12.75tok/s | bastenJul 18, 2026 |
Ornith 1.5 35B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 136.37tok/s | 12.62tok/s | oldwongSep 22, 2026 |
Ornith 1.5 35BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 132.87tok/s | 12.47tok/s | ToshLLM communitySep 10, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 141.28tok/s | 12.44tok/s | oldwongSep 16, 2026 |
Qwen3 Coder 30B-A3BQ2_K · MoE · Signed ToshLLM app · ncmoe 42 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 82.07tok/s | 12.39tok/s | mitchdeSep 29, 2026 |
Qwen3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 49.91tok/s | 12.33tok/s | toshllm@wbig.gsSep 22, 2026 |
Qwen3.8 27BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 620.65tok/s | 12.12tok/s | PixlaAug 29, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 555X4 GB VRAM | GCN / Vega | 72.84tok/s | 12.09tok/s | ToshLLM communitySep 24, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 16 · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 471.86tok/s | 11.84tok/s | ToshLLM communityAug 22, 2026 |
Hy MT2 30BQ6_K · MoE · Signed ToshLLM app · ncmoe 38 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 114.59tok/s | 11.81tok/s | ToshLLM communitySep 1, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 38 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 167.09tok/s | 11.78tok/s | ToshLLM communityJul 24, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 32 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 140.53tok/s | 11.7tok/s | mnerdAug 26, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 157.18tok/s | 11.59tok/s | ToshLLM communitySep 1, 2026 |
Qwen3 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 35 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 154.45tok/s | 11.55tok/s | mnerdJul 31, 2026 |
Hy MT2 30BQ6_K · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 109.04tok/s | 11.53tok/s | ToshLLM communitySep 1, 2026 |
Qwen3.8 27BUD-Q4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | Apple M1 Max51.84 GB VRAM | Apple Silicon | 104.23tok/s | 11.2tok/s | simmc303Sep 22, 2026 |
Gemma 4 26B-A4BUD-Q4_K_XL · MoE · Signed ToshLLM app · ncmoe 22 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 207.43tok/s | 11.19tok/s | mnerdJul 30, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 174.36tok/s | 11.17tok/s | ToshLLM communitySep 2, 2026 |
Deepseek Ai Coder 6.7BQ3_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 40.69tok/s | 11.08tok/s | mitchdeSep 18, 2026 |
Hy MT2 30BQ6_K · MoE · Signed ToshLLM app · ncmoe 43 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 103.56tok/s | 10.88tok/s | ToshLLM communityAug 31, 2026 |
Hy MT2 30BQ6_K · MoE · Signed ToshLLM app · ncmoe 43 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 99.65tok/s | 10.83tok/s | ToshLLM communitySep 1, 2026 |
Qwen3.6 35B-A3BUD-IQ4_XS · MoE · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon VII15.98 GB VRAM | GCN / Vega | 26.84tok/s | 10.63tok/s | 1252er✓Jul 21, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 37 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 105.03tok/s | 10.6tok/s | ToshLLM communitySep 4, 2026 |
Qwen3.8 27BUD-Q8_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo31.98 GB VRAM | GCN / Vega | 126.88tok/s | 10.48tok/s | sedtamensum redditAug 25, 2026 |
Qwen3 Coder 30B-A3BQ2_K · MoE · Signed ToshLLM app · ncmoe 43 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 82.3tok/s | 10.4tok/s | mitchdeSep 25, 2026 |
Qwen3.6 35B-A3BUD-IQ4_XS · MoE · Signed ToshLLM app · ncmoe 10 · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 25.65tok/s | 10.31tok/s | 1252er✓Jul 21, 2026 |
Qwen3.6 35B-A3BUD-IQ4_XS · MoE · Signed ToshLLM app · ncmoe 6 · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 24.35tok/s | 10.23tok/s | 1252er✓Aug 2, 2026 |
Gemma 4 26B-A4BUD-Q6_K · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 113.05tok/s | 10.19tok/s | ToshLLM communitySep 2, 2026 |
Qwen3.6 35B-A3BUD-Q4_K_S · MoE · Signed ToshLLM app · ncmoe 33 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 132.78tok/s | 10.14tok/s | mnerdJul 30, 2026 |
Llama 3.3 70BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega II Duo63.97 GB VRAM | GCN / Vega | 70.36tok/s | 10.01tok/s | warriordr7Oct 8, 2026 |
Gemma 4 26B-A4BUD-Q6_K · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 99.85tok/s | 9.88tok/s | ToshLLM communitySep 1, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 35 · amd-gpu · Metal · comparison | AMD Radeon Pro 5808 GB VRAM | GCN / Vega | 71.18tok/s | 9.83tok/s | ToshLLM communityAug 10, 2026 |
Gemma 4 26B-A4BUD-Q4_K_XL · MoE · Signed ToshLLM app · ncmoe 35 · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 150.24tok/s | 9.71tok/s | mnerdJul 30, 2026 |
Hy MT2 30BQ6_K · MoE · Signed ToshLLM app · ncmoe 43 · amd-gpu · Metal · comparison | AMD Radeon Pro 5600M7.98 GB VRAM | RDNA 1 | 99.37tok/s | 8.93tok/s | ToshLLM communityAug 31, 2026 |
Llama 3.3 70BUD-IQ1_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon PRO W6800X Duo ×2127.94 GB VRAM | RDNA 2 | 291.49tok/s | 8.6tok/s | walrunsAug 22, 2026 |
Qwen3.6 35B-A3BUD-IQ4_XS · MoE · Signed ToshLLM app · ncmoe 25 · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 74.01tok/s | 7.85tok/s | 1252er✓Aug 2, 2026 |
Qwen3.6 35B-A3BQ8_0 · MoE · Signed ToshLLM app · ncmoe 39 · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 38.69tok/s | 7.68tok/s | cloudphreniaSep 22, 2026 |
Gemma 4 12BUD-Q4_K_XL · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro WX 51008 GB VRAM | Unknown | 32.05tok/s | 7.68tok/s | めろんAug 25, 2026 |
DeepSeek Coder V2 Lite InstructIQ2_XS · MoE · Signed ToshLLM app · ncmoe 50 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 92.48tok/s | 7.56tok/s | mitchdeSep 18, 2026 |
Qwen3.8 27BUD-IQ1_S · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5500M7.98 GB VRAM | RDNA 1 | 36.3tok/s | 7.48tok/s | mnerdAug 25, 2026 |
Gemma 4 26B-A4BUD-IQ3_S · MoE · Signed ToshLLM app · ncmoe 3 · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 116.33tok/s | 7.48tok/s | 1252er✓Aug 2, 2026 |
Gemma4 26B-A4BQ8_K · MoE · Signed ToshLLM app · ncmoe 40 · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 46.59tok/s | 5.51tok/s | cloudphreniaSep 22, 2026 |
Qwen3 Coder 30B-A3BQ2_K · MoE · Signed ToshLLM app · ncmoe 48 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 79.58tok/s | 5.24tok/s | mitchdeSep 24, 2026 |
Carnice Qwen3.6 MoE 35BQ3_K_M · MoE · Signed ToshLLM app · ncmoe 3 · amd-gpu · Metal · comparison | AMD Radeon RX Ellesmere Prototype8 GB VRAM | Unknown | 72.63tok/s | 5.05tok/s | 1252er✓Aug 2, 2026 |
Qwen3 Coder 30B-A3BQ2_K · MoE · Signed ToshLLM app · ncmoe 48 · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 82.22tok/s | 4.75tok/s | mitchdeSep 25, 2026 |
Qwen2.5 Coder 7BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 48.27tok/s | 3.55tok/s | mitchdeSep 11, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 48 · amd-gpu · Metal · comparison | AMD Radeon RX 4804 GB VRAM | GCN / Vega | 102.36tok/s | 3.41tok/s | ToshLLM communityOct 12, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro Vega 203.98 GB VRAM | GCN / Vega | 115.63tok/s | 3.27tok/s | ToshLLM communityAug 30, 2026 |
Qwen3 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 5300M3.98 GB VRAM | RDNA 1 | 153.52tok/s | 3.15tok/s | SpidlaDevJul 21, 2026 |
28bbf2fca3be77b1b80b7548f08c80cfe1bfbe08 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 40.64tok/s | 2.91tok/s | mitchdeSep 12, 2026 |
28bbf2fca3be77b1b80b7548f08c80cfe1bfbe08 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 46.54tok/s | 2.91tok/s | mitchdeSep 12, 2026 |
28bbf2fca3be77b1b80b7548f08c80cfe1bfbe08 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon Pro 560X4 GB VRAM | GCN / Vega | 46.12tok/s | 2.91tok/s | mitchdeSep 11, 2026 |
Meta Llama 3.1 8BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon RX 4804 GB VRAM | GCN / Vega | 64.7tok/s | 2.85tok/s | ToshLLM communitySep 10, 2026 |
Gemma 3 4BQ4_K_M · Dense · Signed ToshLLM app · amd-gpu · Metal · comparison | AMD Radeon HD - FirePro D300 ×24 GB VRAM | GCN / Vega | 115.81tok/s | 2.38tok/s | めろんOct 4, 2026 |
Gemma 4 26B-A4BUD-Q4_K_M · MoE · Signed ToshLLM app · ncmoe 45 · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 138.58tok/s | 1.61tok/s | htweedieSep 8, 2026 |
Qwen3 Coder 30B-A3BQ4_K_M · MoE · Signed ToshLLM app · ncmoe 44 · amd-gpu · Metal · comparison | AMD Radeon RX 5808 GB VRAM | GCN / Vega | 135.45tok/s | 1.09tok/s | htweedieSep 8, 2026 |
Qwen3.8 27BQ4_K_M · Dense · Signed ToshLLM app · off · Metal · comparison | AMD Radeon Pro W5500X7.98 GB VRAM | RDNA 1 | 17.54tok/s | 1.07tok/s | ToshLLM communityAug 23, 2026 |
∅
No matching runs
Try another model, GPU, or filter.
Performance comparisons
Compare real-world
performance.
Explore results by GPU and model, with medians, performance ranges and every signed run available for inspection.
By GPU
AMD Radeon Pro 5500M33 runs125.07 tok/s →AMD Radeon RX 58032 runs65.23 tok/s →AMD Radeon Pro 5600M23 runs93.53 tok/s →AMD Radeon RX 6700 XT22 runs250 tok/s →AMD Radeon Pro 560X19 runs179.48 tok/s →AMD Radeon PRO W6800X Duo ×217 runs110.75 tok/s →AMD Radeon Pro Vega II16 runs163.8 tok/s →AMD Radeon Pro 5700 XT13 runs75.93 tok/s →Reproduction protocol
Same test.
Useful comparison.
01
Control the context
Prompt processing uses 512 tokens; generation uses 128. Run three repetitions after a warm-up.
02
Record the machine
GPU, VRAM, quantization, model hash, ToshLLM version and offload settings travel with every result.
03
Preserve provenance
App-signed runs remain labeled as community data. Internally inconsistent or extreme results wait for manual review.
Standard llama-bench command
TOSH_FA_AMD=1 GGML_METAL_CONCURRENCY_DISABLE=1 GGML_SCHED_PREFETCH_EXPERTS=1 GGML_CPU_NO_REPACK=1 ./llama-bench -m MODEL.gguf -ngl 99 --mmap 0 -fa auto -p 512 -n 128 -r 3 --progressMeasurement note: the environment flags match ToshLLM's AMD and MoE runtime. MTP speculative decoding is not supported by llama-bench; measure it with real generation through llama-server. Read the benchmark guide.