Take a look at my definition of the "mathematician's tensor" in the top-level comments. The rank of a tensor is the number of row- and column-vectors you need to feed the tensor to get it to return a real number. AFAIK it's not related to 'rank' in the sense of the rank-nullity theorem.

To move from the mathematician's definition to the ML definition, pick a basis for your row and colum vectors. Now if you want the (i, j, ...) element of the multidimensional array, gather the vectors with one in the i-th position and zero elsewhere, the j-th position and zero elsewhere, etc. Then feed them to the tensor in that order.

It's easiest to see how it works with row vectors (rank (0,1)), column vectors (rank (1, 0)) and matrices (rank (1, 1)) and work from there.