Unfortunately I don't control all of the clients and I'm using self signed certs now. It's just a hassle having to explain to non cs folks how to click past the unsafe prompt which looks to them like an error.
Meanwhile I can hack together a poor man's webrtc by sending raw packets over websockets and playback works just fine. I just lose all the real benefits of webrtc (robustness, udp vs tcp, etc.). It feels like it should be possible to work around "secure context" nonsense since I don't access anything on the clients, they just play back a stream from the devices. But chrome/ff tends to block the ice handshake in http mode.