I completely agree that it's inconvenient as a developer tool, a debug probe is much, much nicer.
I just think that despite all testing and care bugs are still possible, and the ROM bootloader is a backup that's always there. Plop a tiny switch on the PCB, and even if I screw up an OTA update customers will still be able to flash with no special tools (if the device has a USB port, that is).
I also use it as a recovery state for panics, makes the device impossible to brick by a panic loop.