darthrubik/llama-cpp
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

CUDA: fix shared-memory WAR hazard in flash_attn_ext_vec (#30573)
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled

Browse Source 
* CUDA: fix KQ write-after-read race in flash_attn_ext_vec

Assisted-by: ChatGPT

* CUDA: apply review suggestions for KQ sync comments

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
This commit is contained in:
Paul Mairhofer
2026-10-19 07:25:04 +02:00
committed by GitHub
parent 31288c9c66
commit 99db719bdf
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-cuda/fattn-vec.cuh
+3 -1
View File
@@ -317,7 +317,7 @@ static __global__ void flash_attn_ext_vec(
#endif // V_DOT2_F32_F16_AVAILABLE
}
ggml_cuda_syncwarp();
ggml_cuda_syncwarp(); // Needed for KQ synchronization.
#pragma unroll
for (int k0 = 0; k0 < WARP_SIZE; k0 += V_cols_per_iter) {
@@ -374,6 +374,8 @@ static __global__ void flash_attn_ext_vec(
}
#endif // V_DOT2_F32_F16_AVAILABLE
}
ggml_cuda_syncwarp(); // Needed for KQ synchronization.
}
if (sinks && blockIdx.y == 0) {