Correct. Yaroze programs could only run on other Yarozes, plus the worst restriction was that games had to fit entirely in Ram as Yaroze couldn't run burned discs.
of course the bigger problem was that the r3k was too slow. (i ported mpg123 over to run in fixed point, but it still was many times realtime to decode a frame). i wish i'd known about this gte vector unit thing, that may have been a way to make it work... but ah well,
The PS2 had an MDEC too. We used it in one game to do the pop-up talking head dialog between characters mid-gameplay with tiny RAM and CPU use.
i wanted to be able to play mp3 collections from psx units attached to home entertainment systems. there really weren't many other options back then.