Mirror_Repos/ollama_ollama
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

llama: enable FA on CUDA CC 6.x GPUs (#16994)

Browse Source 
Recent upstream Pascal kernel fixes let us compile native SM60/SM61 kernels again instead of relying on PTX JIT, so allow Flash Attention auto at runtime for CC 6.x devices.

Fixes #16591

Fixes #16754
This commit is contained in:
Daniel Hiltgen
2026-07-02 17:11:39 -07:00
committed by GitHub
parent e436db25ff
commit dba1e27fa8
6 changed files with 19 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
llama/server/CMakePresets.json
+1 -1
View File
@@ -68,7 +68,7 @@
"inherits": ["llama_cuda_v12_base"],
"binaryDir": "${sourceDir}/../../build/llama-server-cuda_v12",
"cacheVariables": {
"CMAKE_CUDA_ARCHITECTURES": "50-virtual;52-virtual;60-virtual;61-virtual;70;75;80;86;89;90;90a;120"
"CMAKE_CUDA_ARCHITECTURES": "50-virtual;52-virtual;60;61;70;75;80;86;89;90;90a;120"
}
},
{