| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |     9620.43 ± 279.88 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |      8887.50 ± 49.32 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |        239.18 ± 1.08 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      8594.36 ± 67.44 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |      7918.04 ± 13.14 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |        224.58 ± 0.97 |

build: d2462f8f7 (9590)