Hi, I have read your interesting paper. I recommend you our previous HyperZZW paper (https://arxiv.org/pdf/2401.17948). I think there are a lot of similar concepts here.

1. Context-dependent convolution

2. Global & Local branches

3. Replace large-filter Conv with matrix multiplication

4. Information bottleneck -> Information loss

I also want to share that Mamba is based on the concept of Hyena. And the simplicity is the best (HyperZZW), and Hyena is a failure.