I strongly suggest, for your next step up as a JS person, to take a peek at mediasoup.org. P2P has obvious scalability limitations. That lib not only makes server side forwarding easy (once you read example code), but it abstracts a lot of real-world needed WebRTC features such as simulcast and bandwidth estimation. You'll be making high quality Zoom-like pages in no time (beware though, the dev community ain't great for that lib because at least one of the authors comes off quite rude in the forums).

Also I'd be remiss to not point out the OP's own pion Go libs which do similar on the server side.