All this stuff about warps and streaming multi-processors — is this still the best mental model for targeting modern GPUs with ML workloads? It seems cleaner to think in terms of GEMM, kernels, etc.
And these aren't advanced concepts, it's just the fundamental programming model for GPUs. It's on the level of "Access memory in predictable patterns, ideally sequentially" for CPUs. Everyone knows now that CPUs like arrays and sequential access. And GPUs like interleaved acccess, ideally in sizes no larger than your shared memory.
Kernels are executed at some blocksize, and each block is executed by a SM. The SM partitions each have a limited set of resources, and therefore the number of warps (or waves, AMD) that execute simultaneously should be tuned according to the resources available (registers, shared memory, etc.).
The keyword to search for here is occupancy, and related topics include register pressure/spilling, shared memory and L1 cache-size.
All extremely relevant.