Exactly. If we are going to hypothesize about time travelers then we have to accept that any significant reason to time travel has already happened. Futhermore it has happened more times than we are aware of to create our specific timeline.

Maybe Hitler was a time traveler himself.

I am wondering, that things like killing someone historically significant won't be done unless there is extinction level threat from that individual.

Other wise the people in the future would decide to just let it play.