I could optimize the boot process further, but this is stock Pi OS.

I don't know what librem5 is doing there, but 7s is quite fast for a general purpose os. I wouldn't expect it to get much faster to be honest.