Yet I also use a 7840U in a gaming handheld running Windows, and haven't had any issues there at all. So I think this is related to AMD Linux drivers and/or Wayland. In contrast, my old laptop with an NVIDIA GPU and Xorg has given me zero issues for about a decade now.
So I've decided to just avoid AMD on Linux on my next machine. Intel's upcoming Panther Lake and Nova Lake CPUs seem promising, and their integrated graphics have consistently been improving. I don't think AMD's dominance will continue for much longer.
Make sure it matches the min of the actual spec of the ram that you bought and what the CPU can do.
I used to get crashes like you are describing on a similar machine. The crashes are in the GPU firmware, making debugging a bit of a crap shoot. If you can run windows with the crashing workload on it, you’ll probably find it crashes the same ways as Linux.
For me, it was a bios bug that underclocked the ram. Memory tests, etc passed.
I suspect there are hard performance deadlines in the GPU stack, and the underclocked memory was causing it to miss them, and assume a hang.
If the ram frequency looks OK, check all the hardware configuration knobs you can think of. Something probably auto-detected wrong.
That gave me solid ground for debugging.
But I'll play around with this and the timings, and check if there's a BIOS update that addresses this. Though I still think that AMD's drivers and firmware should be robust enough to support any RAM configuration (within reason), so it would be a problem for them to resolve regardless.
Thanks for the suggestion!
Don't know about transcoding though.