Your analogy is pretty and while I agree with the sentiment I disagree with the scenario presented. I am glad I don't remember everything, yes, clearly perfect memory would be a burden. Rather I am frustrated by my lack of violation, I am not making a conscious choice to remember or forget; the decision is made for me by my flawed existence. The inability to exercise my will and make a meaningful decision over what I remember frustrates me.
That you think life is short is merely because you've forgotten most of it. When you stare at an analog clock, the seconds feel like long eternity.