The academics/experts in the field generally consider 2.1 to be replacement level fertility because:
Accidental deaths before having kids
Gay/Lesbian kids
Kids that “want” to have kids but for whatever reason are unable to
Accidental deaths before having kids
Gay/Lesbian kids
Kids that “want” to have kids but for whatever reason are unable to
Shouldn't "people who don't have kids" be accounted for by lowering the overall measure of "children a woman will give birth to on average"? So if half of women in a country have exactly four kids, and the other half have zero kids for whatever reason, the average number per woman is 2.
Instead, it is calculated by measuring the average number of children born to women at a specific age (e.g. 15, 16..49). Adding all of these up results in the TFR. Thus mortality rates are not included in the TFR.
The opposite is also likely. People may have "accidental" kids, or they could change their mind.
(I still agree that 2.1 is a good approximation, but it is also likely to vary on cultural and natural effects)