So an attacker with the iPhone passcode can lock you out of your Apple account on all devices, even if they don't have your Apple ID password or your Recovery Key. Basically, the iPhone passcode is your only defense if you lose your iPhone. I had always assumed the Apple account password would be needed, and that the passcode is not as important as it is so common for it to be only four or six digits.
I'm going to go setup a stronger passcode now.