With an XOR, if you can control bits on one of the inputs, you can deterministically change one of the bits on the outputs. That is not the case with an HKDF.