Presumably you could just add some random to the timestamps you transmit?
Either way, though, more requests will defeat it one way or another. Whether you can make such an attack impractical will come down to how many requests the attacker can make versus how much noise you can tolerate in the timestamps.
White noise would broaden the distribution of time making the attack harder. A more interesting noise distribution could really obfuscate things by introducing multiple arrival time peaks. A temporally varying multipeak distribution would make life really hard.