Immunity passports reveal a naive understanding of infection. Thousands of people in South Korea have tested positive even after testing negative. Some of those people continued to infect others. One man in China has tested positive for the virus for over 70 days despite having no symptoms. You can read anecdotes on /r/covid19positive from people who, after exercise, suddenly tested positive and developed more severe symptoms after having previously “recovered” and tested negative.
It could very well be that the virus remains reactivatable in some people for an unknown amount of time, maybe even establishing permanent latency like HIV or herpes. Neurological effects are known to occur with this virus and the types of tissue known to be affected continues to expand.
In such a scenario, an immunity “passport” would become more of a scarlet letter.