I'm not an expert on Wittgenstein, but I think the critique of Cantor's diagonal argument is more to do with its implication that an infinite set can be "bigger" than another. To say so is more of a semantics argument than a mathematical one as it takes for granted the meaning of "infinite". That is, if you define an infinite set as being inherently without size, it makes no sense to then assign it comparitive sizes via cardinality.