Signal doesn't store the keys on their servers, nor do they know who is talking to whom. You should read up their protocol.