At least in the US, there's an absolute right against self-incrimination, guaranteed by the Fifth Amendment. The higher courts have been pretty clear, going all the way back to combination locks, that coercing a password would constitute requiring a defendant to incriminate him or herself.
The actions of the judge should be overturned based not only on existing precedent, and principles of fairness and good faith, but also because they violate the privilege against self-incrimination.