For maximum safety you should assume your adversary knows which character set your password comes from. Frankly, there are few reasons not to assume this.