( http://robohub.org/drones-recognise-and-follow-forest-trails... or directly http://ieeexplore.ieee.org/document/7358076/ and probably available on sci-hub )
If we're searching for something that we don't know an approximate location of, depriving everyone around of night vision, making it impossible for helicopters to be nearby and possibly even making thermal imaging devices less useful are disadvantages enough. The only advantage I can see is increased depth and color perception.
If we're searching for something that we know an approximate location of, fair point. Alas, I expect that this is a rarish situation (we either know that the missing person is somewhere in the general area, or within voice range in this particular direction; or we have visual contact).
If we've found something/someone and need to move it/her, we either are in no hurry and need little illumination or we're in a hurry and really care about night vision while carrying the person later or care about being helicopter-friendly (bright light most certainly isn't).
I can't imagine other SAR usecases (other than essentially searching or rescuing).