A nice installer is ideal, but more work to setup I think (I've never figured out how to write an installer so far). And I want the ability to download many versions and put them side by side for eg. running old versions, or bisecting bugs.
In my case the app in question is about ~6MB and an occasionally used utility, I could make a setup version that dynamically loads things... but why? Most users would prefer to just use it, delete it, and if they need it again grab it again.
InnoSetup for the win!
Oh I've had so many nightmares trying to extract the underlying files from Inno Setup installers. 7-Zip doesn't work, innoextract and such is fiddly to find and IIRC doesn't work 100% of the time (forgot).