Hmm well the depth buffer only has accurate depth for opaque objects (and even that's not really true). Things like hair wouldn't be in there. Its not ground truth depth.
It is data, It doesn't have to be ground truth Depth. In fact the difference between what optically appears to be at one depth and the depth map depth is in-itself information. You have described a mechanism that the model can use to identify that what it is supposed to be enhancing is more likely to be hair.