Good questions. A few quick thoughts. These may not be enough though!

First, the mean function specified in the prior is zero for all the points. Thus, points would have a high density near zero.

I have a strong hunch that choosing a "smoother" kernel would not lead to this discontinuity.