Yeah actually, good call. The position of the token matters, since these things use transformers to encode the embeddings.

https://www.assemblyai.com/blog/how-imagen-actually-works/