Not necessarily a problem (for them). The police can serve you a s.49/RIPA notice with a "period for compliance which is reasonable under all the circumstances" (49(4)) so you can go home and get the keys. I expect they'd have grounds to impound your devices, though, if they've grounds for invoking RIPA at all.

If you've genuinely forgotten/don't have the keys, that could be a bigger problem. It is a defence, if charged, not to actually have the key - and they do have to prove it beyond a reasonable doubt if "sufficient evidence of that fact is adduced to raise an issue with respect to it" (53(3)(b)) - but that does ultimately involve convincing a magistrate or a jury.