Indeed - you can even extend this idea to only identify factors less than or equal to the square root - all factors greater than this have a matching factor less than the square root.
50 = 2x5x5 So 100 = 2x2x5x5
You can build your other factors of 4,10, 20 and 25 by multiplying the subsets.