If your GPS antenna is doing any transmitting, you likely have other problems.

If the GPS antenna is separate then you're probably fine leaving it, unless you are worried about IF leakage.

If you rip out the antennas, will all the transmit power emit from the signal traces on the PCB instead?