The negative inference comes from the fact that you can't push an affirmative defence at trial that you haven't hinted at during questioning.

So, if you are arrested for beating up X, and you invoke your right to silence, THEN at trial you put forward a defence of "I wasn't there!" without any substantive evidence to back it up, the prosecutor can say "well, look, if there was an alibi, why didn't you tell the police about it?" The judge is supposed to then instruct the jury that they can draw an inference as to your honesty on the grounds you didn't bring it up during police interview.

The legislation that brings that into force are ss34-39 of the Criminal Justice and Public Order Act 1994.

https://www.legislation.gov.uk/ukpga/1994/33/part/III/crossh...