The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
Basically, it's not as simple as "All the gay participants had rainbow flags in the background of their photo so let's ignore this study"
Well, you've made me laugh and made an excellent point.
I suppose I should read the paper more in-depth.