The kernel not setting the mode basically means no console _at all_, boot log, and similar for you on modern hardware ;=) (or relying one some very very old hacks which might or might not work with your graphic card/screen).

Basically the kernel needs KMS because it want to log things in a readable way during boot etc.

And sure you could argue it's not the kernel doing the logging but e.g. systemd, but it's also not the kernel doing the graphics but your graphics driver. It's just the kernel coordinating it.