The server being or not being secure is only important to the people who operate it. You can examine the client code and see that your messages are encrypted end to end. Signal's entire security model revolves around the idea that you don't need to trust the server.