That's a factor to some extent, but if you're sticking with lossless then the better your compression gets the more your remaining data is dominated by noise.
that's why you need two brains, one to losslessly compress through symbolic expressiveness and another to lossily compress by generating noise like the incompressible noise