Assuming the client is online, you could offload the costly FHE sampling by handing them the raw logits.