In your scenario, yes it is bad if the attacker gets root on your computer, but sealing the key with the TPM means they can't retrieve the key itself. Where the TPM helps is preventing the attacker from establishing low level (kernel, bootloader, or firmware) persistence, since modifications of these components would change the TPM PCR measurements and result in a boot failure.
If the attacker is local and they reset the UEFI, the TPM PCRs are now different and as before, the disk encryption keys will not unseal.
It is also generally recommended to use a pin with TPM, which further complicates this scenario for the attacker because the TPM enforces rate limiting. As the other commenter mentioned, the physical access scenario is commonly a stolen laptop situation, where the attacker would not be in communication with the victim and probably wouldn't return the laptop.