Logit Prisms: Decomposing Transformer Outputs for Mechanistic Interpretability

Custom architecture components break the homogeneity that enables the massive parallelism.

There may be an approach similar to some FPGAs where there are a few things that are commonly needed that might be implemented specifically to avoid constructing them out of generalisable parts. The calculation is quite different though because hardware has more fixed limits and being fixed once manufactured means a great deal more forethought has to take place. Software models, being more flexible can avoid the extra thinking by adding a few million parameters.

One possibility is KANs which can also provide a calculable expression. I could see a fancy convolution KAN being converted into an expression and then applied across an entire input.

I feel like maybe in the future there may be a compiler that can make custom GPU code for specific pathways.

In the short term it might be useful if there is some found formula that requires many layers that can be calculated earlier with a custom expression and fed directly into a higher layer enabling the information to be more broadly used.