I'd really like to get my raspberry pi booting in a couple seconds. I have a few ideas for things I'd like to try but the boot time is too long. Don't need X11 but I would like to use raspbian.
If you do not need the graphical stack, there is room for optimization both in kernel and user space.
On a similar board (4 ARM cores) with Ubuntu I managed to start user space applications in <2 seconds (from power on) and the complete boot (including Xorg/MATE) in <10 seconds.
If you are interested, on Medium I have some write-up (same username, last post).
The Pi 4 is definitely much faster on boot. I wonder how much of your bottleneck is the SD read speed, though.
Yes, I'm getting around 10-15 secs boot time for a headless raspbian. I'm sure it could be made even faster with a lightweight application specific distro.