Combining embeddings is the backbone of multimodal LLMs, such as InstructBLIP[0] or LLaVA[1]. Those architectures take the output tokens from a (frozen) vision transformer and train a very small projection layer between the output token space of the ViT and the input space of the LLM.

[0] https://arxiv.org/abs/2305.06500 [1] https://llava-vl.github.io/