So, the macro is

    \newcommand{\ddim}[3]{\ensuremath{\underset{\scriptscriptstyle #2\times #3}{\rule[-1ex]{0ex}{+1ex}#1}}}

And you use it e.g. like `\ddim {A}NN` to indicate that A is NxN, or `Ax \ddim {=}N1 b` to show that this is an Nx1 equation. I suppose one could fine-tune it, but it was good enough for me.

Looks like this:

https://www.fabian-lischka.de/files/Various/ddim-ex1.png

https://www.fabian-lischka.de/files/Various/ddim-ex2.png