Ask HN: How can I test binary packages automatically?

I have written an OpenSource software (https://speedata.github.io/publisher/index.html) that generates PDF files and I provide ZIP files for various operating systems (Mac/Linux/Windows 64 and 32 bit each). Now from time to time I mess up the ZIP files (such as removing the x flag from the binaries).

I'd like to run a simple test from time to time on each of these operating systems (if possible) that does the following:

1) download a ZIP file from a given location and extract it 2) change to a directory (a hello world example) 3) run a binary from another path within the ZIP file and check if a PDF file has been generated.

Of course I could do a much better check with PDF comparison etc.

How can I automate this task? A "free" online solution would be nice, but any automatic testing solution is appreciated.