This is a good cautionary story of why random parts-swapping can be a waste of time and money. Getting out the DMM and measuring voltages is something fewer and fewer people know how to do when troubleshooting electronics, but it certainly saved the OP here; I'd go a little further and figure out why the monitor seems to be leaking power into its HDMI input when switched off --- possibly an ESD-damaged MOSFET or similar?

The issue does not occur when the monitor is connected via DisplayPort.

https://en.wikipedia.org/wiki/DisplayPort#DP_PWR_(pin_20)

Standard DisplayPort cable connections do not use the DP_PWR pin.

There's also an interesting paragraph there, about some nonstandard cables connecting that pin through.