Yes, the PIN is the least secure way of unlocking a phone, by a wide margin.

If someone can shoulder surf while I unlock my phone with a PIN, then they can watch which buttons I press and now they can unlock my phone too. If I use the fingerprint reader to unlock, then they have no chance.