tests : update Metal fusion baseline for beam (#30764)

* tests: add beam fusion rows to MTL baseline

Regenerate tests/fusion/MTL.csv with test-fusion --record. The newly
added beam architecture has no baseline rows, so test-fusion fails with
2 failing rows (MUL+ADD x6, RMS_NORM+MUL x25). All other arch counts are
unchanged.

Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

* skills: add update-fusion-table skill

Add a skill for regenerating a per-device test-fusion baseline. It keeps
the workflow from tests/fusion/README.md and the parts that are easy to
get wrong: diff the recorded file before applying it, the distinct
meanings of a FAIL row, and never recording a partial run.

Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)
This commit is contained in:
Georgi Gerganov
2026-10-19 09:11:38 +03:00
committed by GitHub
parent 7db0d3d781
commit 27af58168a
2 changed files with 41 additions and 0 deletions
+39
View File
@@ -0,0 +1,39 @@
---
name: update-fusion-table
description: Regenerate a per-device test-fusion baseline (tests/fusion/MTL.csv, ...) after fusion patterns or the dummy-model set change. Use when test-fusion --check fails.
---
# Update a fusion baseline
`tests/fusion/<BACKEND>.csv` is the `test-fusion` golden baseline: `arch,moe,mode,label,count`,
where `count` is how many times a fusion label matched per graph run. See `tests/fusion/README.md`.
Re-record only when the counts moved for the reason the contributor intended. A row failing on
`nmse_fus` (> 1e-4) is a fused-vs-unfused correctness bug, not baseline drift - fix it, never
record it away.
## Steps
```sh
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON # enable the target backend
cmake --build build --config Release --target test-llama-archs --target test-fusion -j
rm -rf build-ci-models && mkdir -p build-ci-models # always regenerate
./build/bin/test-llama-archs -o build-ci-models
D=MTL0; B=tests/fusion/MTL.csv; T=/tmp/$(basename $B) # keep the target backend/device in sync
./build/bin/test-fusion --models build-ci-models --device $D --check $B # read the FAIL rows
./build/bin/test-fusion --models build-ci-models --device $D --record $T
diff $B $T
cp $T $B
./build/bin/test-fusion --models build-ci-models --device $D --check $B # 0 failed, exit 0
```
- Read the diff before applying it. Every added, removed or changed row must be explainable by the
change under review; an unexplained row means the run picked up something unintended.
- `expected 0` in a FAIL row = no baseline row for that label (new arch, or a fusion newly
firing); non-zero `expected` = count changed; `fusion ... is no longer matched` = row stopped
firing.
- `--record` rewrites the whole file, so never record a partial run (`--model FILE`, a filtered
models dir) - it drops the rows of every architecture that was not run.
- `--record` marks every row `ok`; only the `--check` run validates.
+2
View File
@@ -20,6 +20,8 @@ bailingmoe3 ,1 ,any ,GATED_DELTA_NET+CPY , 1
bailingmoe3 ,1 ,any ,MUL+ADD , 2
bailingmoe3 ,1 ,any ,RMS_NORM+MUL , 8
bailingmoe3 ,1 ,any ,RMS_NORM+SCALE , 2
beam ,0 ,any ,MUL+ADD , 6
beam ,0 ,any ,RMS_NORM+MUL , 25
bloom ,0 ,any ,NORM+MUL+ADD , 6
chatglm ,0 ,any ,RMS_NORM+MUL , 5
codeshell ,0 ,any ,NORM+MUL+ADD , 5
1 # test-fusion baseline for device MTL
20 bailingmoe3 ,1 ,any ,MUL+ADD , 2
21 bailingmoe3 ,1 ,any ,RMS_NORM+MUL , 8
22 bailingmoe3 ,1 ,any ,RMS_NORM+SCALE , 2
23 beam ,0 ,any ,MUL+ADD , 6
24 beam ,0 ,any ,RMS_NORM+MUL , 25
25 bloom ,0 ,any ,NORM+MUL+ADD , 6
26 chatglm ,0 ,any ,RMS_NORM+MUL , 5
27 codeshell ,0 ,any ,NORM+MUL+ADD , 5