My understanding of the cause behind the longer-term loss of smell and taste is that it is related to damage to the cells that support your nerve cells. The nerve cells can function for some amount of time without that support, but will eventually start to die off, which causes the sensory loss. This suggests that the symptom could still be caused by damage from the infection, just in a way that has the potential to be a lagging indicator.