You Only Cache Once: Decoder-Decoder Architectures for Language Models

Basically, it is a hybrid model like Jamba.