From my experiment, skipping whole neurons (so whole rows/columns of matrixes) didn’t allow for such good results. In my case 30-50% neurons whole are skipped with 15% effort, but the rest is used partially still.

There are a few papers on a similar theme that a friend sent me today morning - I plan to add them in the citations part