Considering that factorial of 1e6 has about 18e6 bits (and factorial of 1e3 has 8.5e3 bits)? Yes, any factorial that doesn't have a huge amount of bits will be fast enough to calculate that there's not much point to parallelizing it.

as the size of the input to the procedure increases you will indeed be calculating with numbers with a huge amount of bits