My point is that you don"t know yourself, either. You know in any given moment whatever self you have constructed from the sensations, emotions, inferences, and memories available to you. It "s conventional to assume that there is continuity with the selves that you remember, and that"s the default common sense position to take--a sort of pragmatic shorthand--but we don"t actually know that the continuity is real.
In fact, we do know that human memory is spectacularly unreliable, and, moreover, that it is a virtuoso liar, very accomplished at convincing us that we vividly remember things that never actually happened. It's proficient at rearranging memories and sensory impressions into ad hoc stories that we uncritically accept as true records of events.
From this point of view it seems a little over optimistic to me to say that we know ourselves, or to say that I know that the me of today is "the same person" (whatever I might mean by that) as the me of yesterday or last week or 1965.