I can imagine that you can be held liable if, for example, a fire breaks out in your apartment, the apartment owner notices it, tries to enter your apartment to extinguish it, cannot enter because of changed lock, and the building burns down.
[0] as others have mentioned below, this does not seem to be enforceable, even if it is part of the contract, and you are free to change locks after moving in. You only have to give the landlord access to the apartment in case of emergency, and he may enter forcefully if you don't do that (or if you are not present / have changed locks).