I don't have an answer to your particular question but you might have luck posting it at https://softwarerecs.stackexchange.com/ which was made exactly for that kind of question.
I use https://mountainduck.io/ to mount S3 (and Dropbox) as drive. It will still download images to create the thumbnails but I find it reasonably fast.
My team works with lot of images for our ml model. Images are stored in s3. To do manual analysis we need to download the whole folder since s3 do not show the thumbnails or the image.
Is there any app that can do this ?
Well, how do you view them without downloading something? It has to get from S3 to your eyes.
A simple app could be created to do this. Thumbnails require reading the image and making a downsized sample.