DCT is a linear transformation so the model ought to be able to learn it (and typically already does). If you're going to feed in some non-learned transformation it seems like it should be nonlinear to add value.
Still the Transframer results are interesting and has encouraged me to look into it