1) Have you checked what number of the emails bounce? (Incorrect or non existent addresses)

2) Have you thought about skipping password setup and using magic links instead?