Cuda Attention Kernel Patterns
microsoft/onnxruntime/.agents/skills/cuda-attention-kernel-patterns
ai-mlResmi
Resmi Sağlayıcı Skill'iView repo
Patterns and pitfalls for the ONNX domain Attention operator (opset 23/24) CUDA implementation. Use when modifying the dispatch cascade in core/providers/cuda/llm/attention.cc, writing mask/bias CUDA kernels, debugging attention test routing, or adding features to the ONNX Attention op. NOT for contrib domain MultiHeadAttention/GroupQueryAttention.
Dosyalar1 dosya
SKILL.md415 satır
Loading editor…
Kurulum
ÖnerilenTek komut — ajanınız otomatik olarak devreye alır.
Kurulum komutunu görmek için yukarıdan bir AI aracı seçin.
veya
Manuel Kurulum
Daha fazla adımDosyayı indirin ve ajanınızın sistem istemine yapıştırın.
Skill detayları
Versiyonv1.0.0
YazarMicrosoft
Kategoriai-ml
Skill IDmicrosoft/onnxruntime/.agents/skills/cuda-attention-kernel-patterns
İlgili skill'ler
Contrib Op Shape Inference Memory SafetyAudit and fix out-of-range output writes in ONNX Runtime operator shape-inference functions. Use when reviewing or fixing a contrib (or standard) op TypeAndShapeInference where a getNumOutputs() guard precedes a write to a higher output index - optional trailing outputs make a smaller output count schema-valid, so getOutputType(index) can run one past the declared outputs at Graph::Resolve.Cuda Cutlass Fmha Incremental RebuildUse when rebuilding ONNX Runtime CUDA after editing CUTLASS fused-MHA headers (onnxruntime/contrib_ops/cuda/bert/cutlass_fmha/*.h such as kernel_forward.h or fmha_launch_template.h), or when a header edit "passed" an incremental build but test behavior did not change. Explains the nvcc depfile gotcha that produces stale Memory-Efficient-Attention (MEA) kernels and binaries, and how to force a correct recompile. Also covers disk-space frugality on shared GPU dev boxes.Onnx Opset Bump ChecklistStep-by-step checklist for bumping the pinned ONNX dependency / opset in ONNX Runtime (e.g. ONNX 1.21 / opset 26 → 1.22 / opset 27). Use when integrating a new ONNX release or release-candidate, updating the cmake/deps.txt onnx pin or the cmake/external/onnx submodule, regenerating cmake/patches/onnx/onnx.patch, raising kMaxSupportedOpset, or adding a new opset's CPU kernels. Covers the file taxonomy, archive-hash procedures, patch rebase/mirror rules, the RC→formal strategy, and the optimizer/EOrt BuildBuild ONNX Runtime from source. Use this skill when asked to build, compile, or generate CMake files for ONNX Runtime.Ort LintLint and format ONNX Runtime code. Use this skill when asked to lint, format, or check code style for C++ or Python files in ONNX Runtime.Ort TestRun ONNX Runtime tests. Use this skill when asked to run tests, debug test failures, or find and execute specific test cases in ONNX Runtime.