It's nice, but not enough. Modem will still have access to a lot of things, like all unencrypted communication, all metadata, content of all regular calls and SMS, etc.
It's nice that it can't control other devices in the SoC, but that's of little value. Modem, given it's connected over USB can also immitate any other USB device, including keyboards, pointer devices, storage devices, display devices, etc. and actually control the UI.
It can even try to exploit Linux USB stack/drivers by providing specially crafted USB messages to less used/tested USB drivers.
It's somewhat disappointing people repeat this thing about direct memory access as the only way modem can attack the device. It's not nearly enough to use USB. A lot of other mitigations are needed, if you don't trust the modem.