The easy way: compress it with whatever you want (xz? gzip?) and compare the sizes. If it doesn't get significantly smaller, it's probably encrypted or already compressed.