"Human years" makes it sound like you're differentiating from some pseudo timescale like 'dog years'. Man years doesn't have any connotation of sex of the people involved. Ironically the person objecting is the only one projecting a gendered interpretation on to it; it's like punching yourself in the face and complaining that punching people in the faces shouldn't be allowed.
"Man" is just Saxon/Old English for person (and is considered to be from at least PIE origins).
My late mum used to complain about the term Chair instead of Chairman, "I'm a person not a piece of furniture", she was chairman on lots of committees.