> our landlord does have a physical copy of the key to our house. I

This should only be acceptable in the case where your landlord is also your property manager. If they're not, then your letting agent should have a copy of a key.

On the 24 hours note, in 8 years of renting in the UK, I can only remember one instance where I was actually told 24 hours in advance. I was usually emailed at 5pm saying "hey, we have someone coming around for X tomorrow at 9:30, we will let them in". I told them no but they still came every time.