GPU RAM is typically under 32GB (more commonly under 11GB) and quite expensive - for the price of a V100 you could buy 1TB of system DDR4 RAM.
I'm guessing there are disadvantages in memory bandwidth, number of GPU cores, overall FLOPS, but was curious if anyone knew how these pros/cons balance out.
ROCm/HIP/HCC is a broken ecosystem as of today. The 5700XT launched last year and still doesn't have support. I have some faith that they're going to get it straightened out in the next year or so, but today you're just asking for trouble if you're trying to use pytorch or tensorflow without being on team green.
For most ML and GPGPU workloads the bottleneck is memory bandwidth. It’s normal to hit 2TB/s because of all the caches, memory banks and the wide memory bus. I wouldn’t expect the APU to perform nearly as well as even an entry level GPU (say 1660) but I would be curious to see.