Depends on the distro. Changing compilers or compiler flags and any dependent base libraries has a big impact on kernel size, and distros can tune kernel configs for size.

Fitting Linux on a floppy back in the day was about 80% tuning the kernel, and then playing with BusyBox size and finally things like compression or even repacking objects to align better. (It also helped if you had a floppy that could cheat it's way up to 1.68MB, a little known hack that traded reliability for space)