Instead of sorting, just count the occurrences of the distinct values. For 8-bit values, this requires only 256 registers, each with a relatively small number of bits. E.g.: if the maximum matrix size is 16K*16K, then only 14 bits per accumulator is required.

This is just Radix sort and is very easy to implement in digital circuits. It can even reuse the same adder circuits.