The place to enforce USB-PD is at the source, which is what is happening in your described scenario.
Yes, sinks should definitely honour their power contracts, but phones generally already do this (they need to work with 500mA, 2A, and potential faster chargers without overloading them), and don't usually have power-sucking peripherals attached that will mess up their actual draw.
> Not too big of an issue at lower loads, but it will crash if I plug a hard drive in its USB port.
This hard drive probably violates USB spec, as most spinning disks do.