And I seem to recall a recent vulnerability where people were able to bypass the display manager login on some graphical desktop Linux distro.
I just don't feel comfortable shifting the threat to this part of the OS, because desktop Linux has not had the extensive testing required yet. It reminds me of when I was in school and you could bypass NT4 or Windows 98 logins by mashing the Enter key or doing weird user input combinations. Basically manually fuzzing.
If TPM only adds convenience then I'd prefer to continue entering my LUKS password, and my login password, separately.