The Safari of Deep Signal Processing: Hyena and Beyond

Impressively applied here:

https://hazyresearch.stanford.edu/blog/2023-06-29-hyena-dna

Is it reasonable to expect that some companies out there are training LLMs powered by Hyena instead of attention? Like a GPT-3 level size to bet on ultra-long context length?