Don't say it like a police interrogation. Do make it clear there's plenty of time should a candidate need it.
On time, never signal there's a fixed cutoff time until interview over. For the right candidate there shouldn't be. As a manager it's your job to be able to make time.
Not packing stuff back to back without important but non time critical padding's not always an option.
Solutions will depend on your context. I agree pushing back time isn't a valid option, it's not a work culture I believe in.
How about: Using post interview time that should be saved for reflection after the interview §. Trust a colleague to do it or at least start. Make time before you need time by managing it throughout the process.
Or many more options depending on context. Perhaps you're both on flights leaving from the same airport?
§ 'Reflection time' being so important but so often missed. Planning stuff without reflection time is to leadership what assuming the critical path will always be achieved is to project management. Both destined to achieve failure.