Security + build time isn't the issue, but security + dev/testing time is.

Maintaining a secure package of zlib takes linearly more time with more versions of it used.

All distros are manpower limited.