In the US courts are split on the issue. Some say giving an encryption key is testifying, an act of the mind, and you can't force someone to testify against themselves under the 5th. Others say its like handing over a regular key, which you can be forced to do, because the 5th covers testimony, not everything incriminating. It was intended to prevent forced confessions.
Once you're in front of a court you don't get to keep secrets, with the exception of some narrow protections. This has always been the case in the Anglo-American system.