As far as is publicly known, you're not directly vulnerable to having the password broken as long as the device is powered off when it falls into the wrong hands, but there are a lot of caveats. If it's powered on they can possibly extract the password from RAM using a "cold boot attack". They can also freeze the RAM to get its contents to last longer and then moving it to another computer and dump it, recovering your full-disk encryption key.
If the attacker has temporary physical access, they have a number of ways that they can tamper with the device. For example, they can replace the BIOS, the bootloader, the firmware on your keyboard, etc. The new code can record your passwords or send your data to the attacker over the internet.
One other way that you're physically vulnerable is that the attacker can possibly infect your hardware before it even arrives at your home, by replacing hardware while it's shipping. This is probably only available to nation-states.