"the attacker probably won't do that" is very much part of threat modeling, the #1 step in any serious security design.
If an attacker can do it, you must assume they will do it. Because they will. That should be the starting point for any threat model.
"the attacker probably won't read my password through the wall from the radiation off my keyboard"
if your starting point is APT-level adversary then you might as well give up