How much die space ($) will that circuitry, that's probably statistically near zero chance for you main customers workload (who has model weight of 0 or 1!?), add. And, if you can stomach the cost, what else could you put there instead?
What percent of this hardware is running inference for ReLU models? ;)
The die area argument here makes no sense. Supporting structural sparsity can be done either by duplicating the multipliers with and without the support or you have a single general purpose multiplier that does both, in which case you can have twice as many of them.
Also, in ReLU^2 networks, 90%+ parameters are zero.
Any logic you add to the GPU is physical silicon and metal that take up physical space.
> duplicating the multipliers with and without the support or you have a single general purpose multiplier that does both
That would be extra physical logic, which would be extra physical space on the die. "can be done" isn't my point, it's that "doing requires surface area".