darthrubik/llama-cpp
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

0 Releases 8,102 Tags

RSS Feed
  • b11822 d9ee007a06
    Compare

    model : fix qwen4exp layer inputs for DFlash drafters (#30927)

    darthrubik released this 2026-10-22 08:49:38 -05:00 | 5 commits to master since this release

    qwen4exp published the hc-wide residual [n_embd, hc, n_tokens] as
    t_layer_inp, but the drafter reads layer inputs as [n_embd, n_tokens]
    rows: llama_context sizes each embd_layer_inp buffer as n_embd * n_batch
    and the DFlash encoder input is n_extract * n_embd wide. Evaluating a
    DFlash draft over a qwen4exp target therefore aborts on the first request
    in llama_context::extract_layer_inputs.

    Collapse the streams with a plain mean before exposing them, as deepseek4
    already does for its hyper-connection residual, and build the mean only
    for the layers a drafter actually requested.

    Assisted-by: pi:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB)

    Downloads