# llama.cpp Benchmark Results

Generated: 2026-06-13 08:14:45 CEST

## System

| Component | Value |
|---|---|
| GPUs | 2 x NVIDIA RTX PRO 6000 Blackwell Workstation Edition |
| GPU driver | 610.43.02 |
| GPU power limit | 600 W (set via nvidia-smi -pl; persistence mode on) |
| CPU | AMD Ryzen 9 9950X 16-Core Processor |
| System RAM | 128 GiB (OS-visible: 123 GiB) |
| llama.cpp build | version: 9590 (d2462f8f7) built with GNU 14.2.0 for Linux x86_64  |

## Global benchmark configuration

These settings are identical across every model unless noted in a model section.

| Parameter | Value |
|---|---|
| GPU power limit (both GPUs) | 600 W |
| Flash attention | on (`-fa 1`) |
| KV cache type (K and V) | q8_0 (`-ctk q8_0 -ctv q8_0`) |
| CPU threads | 16 (`-t 16`) |
| GPU layers | all (`-ngl 999`) |
| Batch / micro-batch | llama-bench defaults (`-b 2048 -ub 512`) |
| Multi-GPU split (dual) | layer (`-sm layer`) |
| Repetitions | 5 (`-r 5`) |
| Cool-down between models | 300 s |
| nvidia-smi sampling | every 500 ms, scoped to the run's GPUs |

Sweep matrices by placement:
- single card / dual card: `-p 512,4096 -n 128 -d 0,16384`

Column key: `pp` = prefill (tokens/sec), `tg` = decode (tokens/sec);
`@dN` = measurement taken with N tokens already in context.

Note on power readings: `power.draw` samples are instantaneous; NVIDIA's
power controller regulates a time-averaged budget, so individual samples a
few percent above the configured limit are normal transients, not faults.

---

## gpt-oss-120b

- Model spec: `-hf ggml-org/gpt-oss-120b-GGUF`
- Placement: gpu0
- Note: MXFP4 native, ~60 GB, single card

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf ggml-org/gpt-oss-120b-GGUF -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      9504.77 ± 57.55 |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |      9540.90 ± 40.04 |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |        275.07 ± 0.81 |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      7850.25 ± 56.67 |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |      7974.94 ± 13.18 |
| gpt-oss 120B MXFP4 MoE         |  59.02 GiB |   116.83 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |        235.14 ± 0.41 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 56 | N/A | 581.7 | 59.9 | none |

Logs: `bench_logs/600/gpt-oss-120b.log` | thermals: `bench_logs/600/gpt-oss-120b.nvidia-smi.csv`

---

## qwen3-coder-next-80b

- Model spec: `-hf unsloth/Qwen3-Coder-Next-GGUF:Q8_0`
- Placement: gpu0
- Note: 80B-A3B, Q8 ~85 GB, single card

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf unsloth/Qwen3-Coder-Next-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen3next 80B.A3B Q8_0         |  78.98 GiB |    79.67 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      4869.04 ± 19.60 |
| qwen3next 80B.A3B Q8_0         |  78.98 GiB |    79.67 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       4886.82 ± 6.52 |
| qwen3next 80B.A3B Q8_0         |  78.98 GiB |    79.67 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |        195.07 ± 0.84 |
| qwen3next 80B.A3B Q8_0         |  78.98 GiB |    79.67 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      4628.00 ± 21.86 |
| qwen3next 80B.A3B Q8_0         |  78.98 GiB |    79.67 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       4611.51 ± 7.59 |
| qwen3next 80B.A3B Q8_0         |  78.98 GiB |    79.67 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |        183.41 ± 1.82 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 56 | N/A | 467.1 | 80.0 | none |

Logs: `bench_logs/600/qwen3-coder-next-80b.log` | thermals: `bench_logs/600/qwen3-coder-next-80b.nvidia-smi.csv`

---

## qwen2.5-coder-32b

- Model spec: `-hf unsloth/Qwen2.5-Coder-32B-Instruct-GGUF:Q8_0`
- Placement: gpu0
- Note: 32B dense, Q8 ~35 GB

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf unsloth/Qwen2.5-Coder-32B-Instruct-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen2 32B Q8_0                 |  32.42 GiB |    32.76 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |     3977.05 ± 330.93 |
| qwen2 32B Q8_0                 |  32.42 GiB |    32.76 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       3849.90 ± 7.68 |
| qwen2 32B Q8_0                 |  32.42 GiB |    32.76 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         43.37 ± 0.02 |
| qwen2 32B Q8_0                 |  32.42 GiB |    32.76 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      2656.02 ± 28.25 |
| qwen2 32B Q8_0                 |  32.42 GiB |    32.76 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       2479.68 ± 5.20 |
| qwen2 32B Q8_0                 |  32.42 GiB |    32.76 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         38.60 ± 0.02 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 71 | N/A | 606.9 | 35.3 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/qwen2.5-coder-32b.log` | thermals: `bench_logs/600/qwen2.5-coder-32b.nvidia-smi.csv`

---

## devstral-small-2-24b

- Model spec: `-hf unsloth/Devstral-Small-2507-GGUF:Q8_0`
- Placement: gpu0
- Note: 24B dense, Q8 ~26 GB (verify repo)

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf unsloth/Devstral-Small-2507-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| llama 13B Q8_0                 |  23.33 GiB |    23.57 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |     5844.49 ± 223.42 |
| llama 13B Q8_0                 |  23.33 GiB |    23.57 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       5718.36 ± 8.61 |
| llama 13B Q8_0                 |  23.33 GiB |    23.57 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         61.37 ± 0.03 |
| llama 13B Q8_0                 |  23.33 GiB |    23.57 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      4706.09 ± 68.28 |
| llama 13B Q8_0                 |  23.33 GiB |    23.57 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       4460.59 ± 5.32 |
| llama 13B Q8_0                 |  23.33 GiB |    23.57 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         56.17 ± 0.03 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 65 | N/A | 602.5 | 25.2 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/devstral-small-2-24b.log` | thermals: `bench_logs/600/devstral-small-2-24b.nvidia-smi.csv`

---

## qwen3-235b-a22b

- Model spec: `-hf unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF:Q4_K_M`
- Placement: dual
- Note: 235B-A22B, Q4 ~133 GB, dual

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0,1 |
| Split mode | -sm layer |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0,1 llama-bench -hf unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF:Q4_K_M -ngl 999 -sm layer -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen3moe 235B.A22B Q4_K - Medium | 132.39 GiB |   235.09 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |     1976.68 ± 254.95 |
| qwen3moe 235B.A22B Q4_K - Medium | 132.39 GiB |   235.09 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |      2613.35 ± 13.77 |
| qwen3moe 235B.A22B Q4_K - Medium | 132.39 GiB |   235.09 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         83.92 ± 0.12 |
| qwen3moe 235B.A22B Q4_K - Medium | 132.39 GiB |   235.09 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |       1587.82 ± 6.95 |
| qwen3moe 235B.A22B Q4_K - Medium | 132.39 GiB |   235.09 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       1861.49 ± 4.90 |
| qwen3moe 235B.A22B Q4_K - Medium | 132.39 GiB |   235.09 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         53.60 ± 0.05 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 68 | N/A | 478.1 | 69.1 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |
| 1 | 63 | N/A | 438.0 | 67.0 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/qwen3-235b-a22b.log` | thermals: `bench_logs/600/qwen3-235b-a22b.nvidia-smi.csv`

---

## minimax-m2.7

- Model spec: `-hf unsloth/MiniMax-M2.7-GGUF:UD-Q4_K_XL`
- Placement: dual
- Note: ~230B-A10B, Q4, dual (verify tag)

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0,1 |
| Split mode | -sm layer |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0,1 llama-bench -hf unsloth/MiniMax-M2.7-GGUF:UD-Q4_K_XL -ngl 999 -sm layer -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| minimax-m2 230B.A10B Q4_K - Medium | 131.10 GiB |   228.69 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |     1836.52 ± 596.82 |
| minimax-m2 230B.A10B Q4_K - Medium | 131.10 GiB |   228.69 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       3726.95 ± 3.51 |
| minimax-m2 230B.A10B Q4_K - Medium | 131.10 GiB |   228.69 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |        129.48 ± 0.45 |
| minimax-m2 230B.A10B Q4_K - Medium | 131.10 GiB |   228.69 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |       1945.03 ± 5.11 |
| minimax-m2 230B.A10B Q4_K - Medium | 131.10 GiB |   228.69 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       2777.59 ± 4.18 |
| minimax-m2 230B.A10B Q4_K - Medium | 131.10 GiB |   228.69 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         89.62 ± 0.24 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 68 | N/A | 569.3 | 69.0 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |
| 1 | 65 | N/A | 540.5 | 66.2 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/minimax-m2.7.log` | thermals: `bench_logs/600/minimax-m2.7.nvidia-smi.csv`

---

## glm-4.6-355b

- Model spec: `-hf unsloth/GLM-4.6-GGUF:UD-Q2_K_XL`
- Placement: dual
- Note: 355B-A32B, Q2_K_XL ~135 GB, dual

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0,1 |
| Split mode | -sm layer |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0,1 llama-bench -hf unsloth/GLM-4.6-GGUF:UD-Q2_K_XL -ngl 999 -sm layer -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| glm4moe 355B.A32B Q2_K - Medium | 125.43 GiB |   356.79 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      960.36 ± 108.47 |
| glm4moe 355B.A32B Q2_K - Medium | 125.43 GiB |   356.79 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       1178.30 ± 4.56 |
| glm4moe 355B.A32B Q2_K - Medium | 125.43 GiB |   356.79 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         61.15 ± 0.08 |
| glm4moe 355B.A32B Q2_K - Medium | 125.43 GiB |   356.79 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |        741.40 ± 3.10 |
| glm4moe 355B.A32B Q2_K - Medium | 125.43 GiB |   356.79 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |        844.63 ± 2.44 |
| glm4moe 355B.A32B Q2_K - Medium | 125.43 GiB |   356.79 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         37.84 ± 0.03 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 78 | N/A | 481.7 | 64.0 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |
| 1 | 71 | N/A | 496.1 | 65.7 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/glm-4.6-355b.log` | thermals: `bench_logs/600/glm-4.6-355b.nvidia-smi.csv`

---

## qwen3-coder-480b

- Model spec: `-hf unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF:UD-Q2_K_XL`
- Placement: dual
- Note: 480B-A35B, Q2 ~180 GB, dual (tight)

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0,1 |
| Split mode | -sm layer |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0,1 llama-bench -hf unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF:UD-Q2_K_XL -ngl 999 -sm layer -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen3moe ?B Q2_K - Medium      | 167.91 GiB |   480.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      928.16 ± 134.16 |
| qwen3moe ?B Q2_K - Medium      | 167.91 GiB |   480.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       1443.73 ± 4.32 |
| qwen3moe ?B Q2_K - Medium      | 167.91 GiB |   480.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         73.36 ± 0.08 |
| qwen3moe ?B Q2_K - Medium      | 167.91 GiB |   480.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |        868.21 ± 6.64 |
| qwen3moe ?B Q2_K - Medium      | 167.91 GiB |   480.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       1129.37 ± 4.73 |
| qwen3moe ?B Q2_K - Medium      | 167.91 GiB |   480.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         48.91 ± 0.07 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 81 | N/A | 558.5 | 87.7 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |
| 1 | 69 | N/A | 469.1 | 84.5 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/qwen3-coder-480b.log` | thermals: `bench_logs/600/qwen3-coder-480b.nvidia-smi.csv`

---

## qwen3.6-27b

- Model spec: `-hf unsloth/Qwen3.6-27B-GGUF:Q8_0`
- Placement: gpu0
- Note: 27B dense, Q8 ~30 GB, single card

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf unsloth/Qwen3.6-27B-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen35 27B Q8_0                |  26.62 GiB |    26.90 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |     4276.26 ± 255.04 |
| qwen35 27B Q8_0                |  26.62 GiB |    26.90 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       4298.55 ± 7.49 |
| qwen35 27B Q8_0                |  26.62 GiB |    26.90 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         50.45 ± 0.04 |
| qwen35 27B Q8_0                |  26.62 GiB |    26.90 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      3756.93 ± 58.75 |
| qwen35 27B Q8_0                |  26.62 GiB |    26.90 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       3677.07 ± 5.56 |
| qwen35 27B Q8_0                |  26.62 GiB |    26.90 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         48.90 ± 0.04 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 70 | N/A | 603.0 | 27.3 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/qwen3.6-27b.log` | thermals: `bench_logs/600/qwen3.6-27b.nvidia-smi.csv`

---

## qwen3.6-35b-a3b

- Model spec: `-hf unsloth/Qwen3.6-35B-A3B-GGUF:Q8_0`
- Placement: gpu0
- Note: 35B-A3B MoE, Q8 ~38 GB, single card

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      9919.34 ± 56.52 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |      9810.17 ± 45.94 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |        239.19 ± 1.02 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      8635.87 ± 61.69 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       8755.26 ± 8.95 |
| qwen35moe 35B.A3B Q8_0         |  34.36 GiB |    34.66 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |        224.58 ± 0.95 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 56 | N/A | 540.5 | 35.3 | none |

Logs: `bench_logs/600/qwen3.6-35b-a3b.log` | thermals: `bench_logs/600/qwen3.6-35b-a3b.nvidia-smi.csv`

---

## hermes-4.3-36b

- Model spec: `-hf NousResearch/Hermes-4.3-36B-GGUF:Q8_0`
- Placement: gpu0
- Note: 36B dense (Seed-OSS-36B), Q8 ~38 GB, single card

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf NousResearch/Hermes-4.3-36B-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| seed_oss 36B Q8_0              |  35.78 GiB |    36.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      3638.58 ± 29.01 |
| seed_oss 36B Q8_0              |  35.78 GiB |    36.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       3281.83 ± 6.47 |
| seed_oss 36B Q8_0              |  35.78 GiB |    36.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         39.52 ± 0.01 |
| seed_oss 36B Q8_0              |  35.78 GiB |    36.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      1869.72 ± 17.69 |
| seed_oss 36B Q8_0              |  35.78 GiB |    36.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       1724.67 ± 5.57 |
| seed_oss 36B Q8_0              |  35.78 GiB |    36.15 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         32.06 ± 0.01 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 74 | N/A | 604.6 | 38.6 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/hermes-4.3-36b.log` | thermals: `bench_logs/600/hermes-4.3-36b.nvidia-smi.csv`

---

## hermes-4-70b

- Model spec: `-hf unsloth/Hermes-4-70B-GGUF:Q8_0`
- Placement: gpu0
- Note: 70B dense (Llama 3.1), Q8 ~75 GB, single card

Parameters for this run:

| Parameter | Value |
|---|---|
| GPU power limit | 600 W (both GPUs) |
| Visible GPUs | 0 |
| Split mode | n/a (single GPU) |
| CPU-MoE offload | none |
| Flash attention | on |
| KV cache (K/V) | q8_0 |
| Threads | 16 |
| Prompt sizes | 512,4096 |
| Gen tokens | 128 |
| Depths | 0,16384 |
| Repetitions | 5 |

Exact command:

```bash
CUDA_VISIBLE_DEVICES=0 llama-bench -hf unsloth/Hermes-4-70B-GGUF:Q8_0 -ngl 999 -fa 1 -ctk q8_0 -ctv q8_0 -t 16 -p 512,4096 -n 128 -d 0,16384 -r 5 -o md
```

### llama-bench results

| model                          |       size |     params | backend    | ngl | type_k | type_v |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | -----: | --: | --------------: | -------------------: |
| llama 70B Q8_0                 |  69.82 GiB |    70.55 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           pp512 |      2005.68 ± 10.63 |
| llama 70B Q8_0                 |  69.82 GiB |    70.55 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |          pp4096 |       1915.20 ± 6.61 |
| llama 70B Q8_0                 |  69.82 GiB |    70.55 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |           tg128 |         20.77 ± 0.01 |
| llama 70B Q8_0                 |  69.82 GiB |    70.55 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  pp512 @ d16384 |      1464.24 ± 11.31 |
| llama 70B Q8_0                 |  69.82 GiB |    70.55 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 | pp4096 @ d16384 |       1383.69 ± 4.52 |
| llama 70B Q8_0                 |  69.82 GiB |    70.55 B | CUDA       | 999 |   q8_0 |   q8_0 |   1 |  tg128 @ d16384 |         18.44 ± 0.01 |

build: d2462f8f7 (9590)

### Thermals and power (sampled during the run)

| GPU | Max core temp (C) | Max mem temp (C) | Max power draw (W) | Max VRAM used (GiB) | Throttle reasons observed |
|---|---|---|---|---|---|
| 0 | 79 | N/A | 606.8 | 73.0 | 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004 0x0000000000000004  |

Logs: `bench_logs/600/hermes-4-70b.log` | thermals: `bench_logs/600/hermes-4-70b.nvidia-smi.csv`

---

## Summary (tokens/sec)

Extracted from each model's llama-bench table. `pp` = prefill, `tg` = decode.

| Model | Test | tokens/sec |
|---|---|---|
| gpt-oss-120b | pp512 | 9504.77 ± 57.55 |
| gpt-oss-120b | pp4096 | 9540.90 ± 40.04 |
| gpt-oss-120b | tg128 | 275.07 ± 0.81 |
| gpt-oss-120b | pp512 @ d16384 | 7850.25 ± 56.67 |
| gpt-oss-120b | pp4096 @ d16384 | 7974.94 ± 13.18 |
| gpt-oss-120b | tg128 @ d16384 | 235.14 ± 0.41 |
| qwen3-coder-next-80b | pp512 | 4869.04 ± 19.60 |
| qwen3-coder-next-80b | pp4096 | 4886.82 ± 6.52 |
| qwen3-coder-next-80b | tg128 | 195.07 ± 0.84 |
| qwen3-coder-next-80b | pp512 @ d16384 | 4628.00 ± 21.86 |
| qwen3-coder-next-80b | pp4096 @ d16384 | 4611.51 ± 7.59 |
| qwen3-coder-next-80b | tg128 @ d16384 | 183.41 ± 1.82 |
| qwen2.5-coder-32b | pp512 | 3977.05 ± 330.93 |
| qwen2.5-coder-32b | pp4096 | 3849.90 ± 7.68 |
| qwen2.5-coder-32b | tg128 | 43.37 ± 0.02 |
| qwen2.5-coder-32b | pp512 @ d16384 | 2656.02 ± 28.25 |
| qwen2.5-coder-32b | pp4096 @ d16384 | 2479.68 ± 5.20 |
| qwen2.5-coder-32b | tg128 @ d16384 | 38.60 ± 0.02 |
| devstral-small-2-24b | pp512 | 5844.49 ± 223.42 |
| devstral-small-2-24b | pp4096 | 5718.36 ± 8.61 |
| devstral-small-2-24b | tg128 | 61.37 ± 0.03 |
| devstral-small-2-24b | pp512 @ d16384 | 4706.09 ± 68.28 |
| devstral-small-2-24b | pp4096 @ d16384 | 4460.59 ± 5.32 |
| devstral-small-2-24b | tg128 @ d16384 | 56.17 ± 0.03 |
| qwen3-235b-a22b | pp512 | 1976.68 ± 254.95 |
| qwen3-235b-a22b | pp4096 | 2613.35 ± 13.77 |
| qwen3-235b-a22b | tg128 | 83.92 ± 0.12 |
| qwen3-235b-a22b | pp512 @ d16384 | 1587.82 ± 6.95 |
| qwen3-235b-a22b | pp4096 @ d16384 | 1861.49 ± 4.90 |
| qwen3-235b-a22b | tg128 @ d16384 | 53.60 ± 0.05 |
| minimax-m2.7 | pp512 | 1836.52 ± 596.82 |
| minimax-m2.7 | pp4096 | 3726.95 ± 3.51 |
| minimax-m2.7 | tg128 | 129.48 ± 0.45 |
| minimax-m2.7 | pp512 @ d16384 | 1945.03 ± 5.11 |
| minimax-m2.7 | pp4096 @ d16384 | 2777.59 ± 4.18 |
| minimax-m2.7 | tg128 @ d16384 | 89.62 ± 0.24 |
| glm-4.6-355b | pp512 | 960.36 ± 108.47 |
| glm-4.6-355b | pp4096 | 1178.30 ± 4.56 |
| glm-4.6-355b | tg128 | 61.15 ± 0.08 |
| glm-4.6-355b | pp512 @ d16384 | 741.40 ± 3.10 |
| glm-4.6-355b | pp4096 @ d16384 | 844.63 ± 2.44 |
| glm-4.6-355b | tg128 @ d16384 | 37.84 ± 0.03 |
| qwen3-coder-480b | pp512 | 928.16 ± 134.16 |
| qwen3-coder-480b | pp4096 | 1443.73 ± 4.32 |
| qwen3-coder-480b | tg128 | 73.36 ± 0.08 |
| qwen3-coder-480b | pp512 @ d16384 | 868.21 ± 6.64 |
| qwen3-coder-480b | pp4096 @ d16384 | 1129.37 ± 4.73 |
| qwen3-coder-480b | tg128 @ d16384 | 48.91 ± 0.07 |
| qwen3.6-27b | pp512 | 4276.26 ± 255.04 |
| qwen3.6-27b | pp4096 | 4298.55 ± 7.49 |
| qwen3.6-27b | tg128 | 50.45 ± 0.04 |
| qwen3.6-27b | pp512 @ d16384 | 3756.93 ± 58.75 |
| qwen3.6-27b | pp4096 @ d16384 | 3677.07 ± 5.56 |
| qwen3.6-27b | tg128 @ d16384 | 48.90 ± 0.04 |
| qwen3.6-35b-a3b | pp512 | 9919.34 ± 56.52 |
| qwen3.6-35b-a3b | pp4096 | 9810.17 ± 45.94 |
| qwen3.6-35b-a3b | tg128 | 239.19 ± 1.02 |
| qwen3.6-35b-a3b | pp512 @ d16384 | 8635.87 ± 61.69 |
| qwen3.6-35b-a3b | pp4096 @ d16384 | 8755.26 ± 8.95 |
| qwen3.6-35b-a3b | tg128 @ d16384 | 224.58 ± 0.95 |
| hermes-4.3-36b | pp512 | 3638.58 ± 29.01 |
| hermes-4.3-36b | pp4096 | 3281.83 ± 6.47 |
| hermes-4.3-36b | tg128 | 39.52 ± 0.01 |
| hermes-4.3-36b | pp512 @ d16384 | 1869.72 ± 17.69 |
| hermes-4.3-36b | pp4096 @ d16384 | 1724.67 ± 5.57 |
| hermes-4.3-36b | tg128 @ d16384 | 32.06 ± 0.01 |
| hermes-4-70b | pp512 | 2005.68 ± 10.63 |
| hermes-4-70b | pp4096 | 1915.20 ± 6.61 |
| hermes-4-70b | tg128 | 20.77 ± 0.01 |
| hermes-4-70b | pp512 @ d16384 | 1464.24 ± 11.31 |
| hermes-4-70b | pp4096 @ d16384 | 1383.69 ± 4.52 |
| hermes-4-70b | tg128 @ d16384 | 18.44 ± 0.01 |

Notes on interpretation:
- Decode (`tg`) is bounded by active parameters and memory bandwidth; MoE models
  decode far faster than dense models of similar total size.
- The second GPU adds capacity and context; do not expect dual-card `tg` to be ~2x
  a single card on this PCIe (no-NVLink) system.
- Single-card (placement `gpu0`) and dual-card runs use different hardware budgets;
  their absolute throughput numbers are not directly comparable.
- If `Max mem temp` reads `N/A`, the driver/board did not expose the GDDR7 sensor.
- A throttle reason other than `none` means the card hit a limit during the run.
  `SW power cap` is expected whenever the power limit is set below the card's max.