I believe that density is 100x to 1000x more important for fast experience and computational cost than the near instantaneous decoding speed of common lossless formats.
As for which is more important out of encode and decode speed, I would personally rank decode speed higher. For example, app icons on your home screen were encoded once (by the graphic artist) but are decoded zillions of times (by every user every day).
We have ideas how to make it faster than WebP lossless, but haven't engaged with that yet -- the lossless decoding is already fast enough not to be a blocker.
Also, given how efficient and quality-sure the lossy is, it will become tempting to migrate many of the PNG/GIF/WebP lossless cases to JPEG XL lossy further reducing the priority in JPEG XL lossless. This can be a 4x-5x savings whereas a better lossless is only 10-40 % savings in density.