> completely offtopic but is rolling with rocm worth it?

It's so fucking easy.

From AMD: https://lemonade-server.ai/

Then you can easily throw a openweb-ui container in front, and then connect to the openweb-ui via your mobile app of choice (if you want chat, otherwise you just point your harness of choice at the lemonade server api endpoint).