Why are you assuming that a determined attacker doesn't control your L4 stack? MITMs are a threat, your network could be compromised, routers (especially consumer routers) are rife with vulnerabilities. This is the entire reason "zero trust" is pushed.