Considering the plethora of ARM trustzone and bootloader attacks against Android devices I don't think an abnormal skilled attacker means what you think it means the average repair shop can extract data from the majority of android phones today.
You also have successful key recovery/bypass attacks against most non-hardware backed crypto Android devices as well.
Like Thomas said it's not impossible to build a secure device, it's not even impossible to build a secure open mobile device but it doesn't seem that they are doing it.
Their focus is on having feature parity, using commodity hardware and just having an FSF approved stack.
Having an FSF approved stack doesn't make you more secure by default.
And usability has a great impact on security, I remember the early android days where getting a file off the device was PITA so myself and many others were running an FTP server on the phone, and since most of our phones were rooted im pretty sure it was running as root.
The other side is things like permissions while mobile operating systems aren't that great still they've began taking application permissions really seriously going through the Librem documentation I don't see anything that is even remotely close to the level of granularity that Android and iOS offer today.
Sure they might add that in the future but the point being is still that there is little chance that the first phone they launch would be more secure than an android phone yet alone a modern iPhone in fact I would bet at least one paycheck that they would be considerably less secure at least initially, and then it's the question of how they would be able to maintain and support their platform given their size to begin with.
I don't doubt the intentions of the developers I just highly doubt that anything they set is even remotely achievable.