Interactive dialogue · 2026-10-08互动对话 · 2026-10-08
GPU Kernel Tips: From Warps to Tile SizesGPU 算子的一些 tips：从 warp 到 tile size
What I picked up about GPU kernels from talking with other people, written as a dialogue tree: from coalescing and bank conflicts to warpgroups, a few hardware numbers that settle FlashAttention's block_KV, and finally FlashFFN and its extension to MoE. Open the follow-ups you care about and skip the rest.和别人交流时学到的 GPU kernel 知识，整理成一棵对话树：从 coalesce 和 bank conflict 讲到 warpgroup，再用几个硬件数字推出 FlashAttention 的 block_KV，最后是 FlashFFN 和它在 MoE 上的拓展。想深挖的追问就点开，其余的跳过。
Read the article →阅读全文 →
