I wonder how multimodal input and output will work with the chat API endpoints. I assume the messages array will contain URLs to an image, or maybe base64 encoded image data or something.

Maybe it will not be called the Chat API but rather the Multimodal API.