mikhail/llama.cpp-mtp-turboquant
1
0
Fork 0
Code Issues Pull Requests Actions 29 Packages Projects Releases Wiki Activity
Files
5fa07c2f93c73161bf09ef0b23b5d2686f9a073e
llama.cpp-mtp-turboquant/ggml/src/ggml-cuda/template-instances
T
History
Johannes Gäßler 5fa07c2f93 CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
..
fattn-mma-f16-instance-ncols1_1-ncols2_8.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_2-ncols2_4.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_2-ncols2_8.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_4-ncols2_2.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_4-ncols2_4.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_4-ncols2_8.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_8-ncols2_1.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_8-ncols2_2.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_8-ncols2_4.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_8-ncols2_8.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_16-ncols2_1.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_16-ncols2_2.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_16-ncols2_4.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_32-ncols2_1.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_32-ncols2_2.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-mma-f16-instance-ncols1_64-ncols2_1.cu
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
fattn-vec-f16-instance-hs64-f16-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs64-f16-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs64-f16-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs64-f16-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs64-f16-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs64-f16-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-f16-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-f16-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-f16-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-f16-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-f16-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-f16-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_0-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_0-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_0-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_0-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_0-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_0-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_1-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_1-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_1-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_1-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_1-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q4_1-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_0-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_0-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_0-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_0-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_0-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_0-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_1-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_1-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_1-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_1-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_1-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q5_1-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q8_0-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q8_0-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q8_0-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q8_0-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q8_0-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs128-q8_0-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f16-instance-hs256-f16-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs64-f16-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs64-f16-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs64-f16-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs64-f16-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs64-f16-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs64-f16-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-f16-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-f16-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-f16-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-f16-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-f16-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-f16-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_0-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_0-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_0-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_0-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_0-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_0-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_1-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_1-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_1-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_1-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_1-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q4_1-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_0-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_0-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_0-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_0-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_0-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_0-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_1-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_1-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_1-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_1-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_1-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q5_1-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q8_0-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q8_0-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q8_0-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q8_0-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q8_0-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs128-q8_0-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
fattn-vec-f32-instance-hs256-f16-f16.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
generate_cu_files.py
CUDA: optimize FA for GQA + large batches (#12014)
2025-02-22 12:20:17 +01:00
mmq-instance-iq1_s.cu
CUDA: MMQ code deduplication + iquant support (#8495)
2024-07-20 22:25:26 +02:00
mmq-instance-iq2_s.cu
CUDA: MMQ code deduplication + iquant support (#8495)
2024-07-20 22:25:26 +02:00
mmq-instance-iq2_xs.cu
CUDA: MMQ code deduplication + iquant support (#8495)
2024-07-20 22:25:26 +02:00
mmq-instance-iq2_xxs.cu
CUDA: MMQ code deduplication + iquant support (#8495)
2024-07-20 22:25:26 +02:00
mmq-instance-iq3_s.cu
CUDA: MMQ code deduplication + iquant support (#8495)
2024-07-20 22:25:26 +02:00
mmq-instance-iq3_xxs.cu
CUDA: MMQ code deduplication + iquant support (#8495)
2024-07-20 22:25:26 +02:00
mmq-instance-iq4_nl.cu
CUDA: MMQ support for iq4_nl, iq4_xs (#8278)
2024-07-05 09:06:31 +02:00
mmq-instance-iq4_xs.cu
CUDA: MMQ support for iq4_nl, iq4_xs (#8278)
2024-07-05 09:06:31 +02:00
mmq-instance-q2_k.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q3_k.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q4_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q4_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q4_k.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q5_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q5_1.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q5_k.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q6_k.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00
mmq-instance-q8_0.cu
llama : reorganize source code + improve CMake (#8006)
2024-06-26 18:33:02 +03:00