I think the better solution would to be very explicit and clear about what your app does. Stating that it creates a "My Downloads" folder and just puts files into that folder, and nothing more, should be enough to assuage fears.