Once matured, kernels tend to never break backwards compatibility (or try very hard not to), and rarely track bug fixes with individual numbering. Often many changes are lumped into a single version update, because so many changes are already happening across many different subsystems.
I suppose also, now that I think about it, Semantic Versioning works much better for smaller scoped and smaller scale projects (like well scoped and designed libraries). The kernel is such a large piece of software almost like a collection of libraries hidden behind a single set of matures APIs that incrementing counters for every bug fix or tiny change doesn't make sense---and as has been established, major numbers would just never bump.