Hmm...Yes. But every time a reply is made? No! Only when there are new unreads since the last time I read the replies.

And make sure this can be turned off in settings.