This is correct. The problem in this example is not the pendrive, but the automatic selection of device drivers based on the PC OS 100% trusting physical access.

An easy way to see this is recompile the USB keyboard driver to ignore keyboard descriptors with a particular address or vendor ID. If you do this, the pendrive can't do anything because USB is host-controlled, as implemented by the OS. Without the OS initiating a conversation with the pendrive and saying "ok, I'll configure you as a keyboard and interpret responses from you as keystrokes", it can't happen.

As you point out, the main CPU on a phone does not implement HID autoconfig on the internal baseband bus.