That could become an automated improvement as well... although it was probably published in some paper 40 years ago and I just don't know the name of it.

Basically a second-pass where you look for outliers in each character-group, reassign them to another group, and keep the change if it improves similarity-scores the character groups involved. Then iterate a bit until nothing seems to be improving.

For example, one might find "the 3 that is least-like all the other 3s", temporarily reassign it to "8", and then keep that change if it means an improvement in the the scores for "how closely all sub-pictures of 3s resemble each other" and "how closely all sub-pictures 8s resemble each other".

That might backfire if a document has different typefaces in it though, where it makes the mistake of putting all "3"s from different typefaces together, ruining the group-similarity scores.