Great post, one nitpick -- I wouldn't say that a matrix is a "sparsely defined" function, but rather a function defined on a finite grid. It might also be worth pointing out that same approach works for any graph, not just a grid.
The author uses a matrix quite differently. You give it two integer coordinates i and j and it gives you the value at position (i, j) back. That's a valid use, but not quite what you'd expect in a math-oriented article.
In linear algebra, we would interpret this as a linear map. A true equation would be f([1, 2, 3]^T) = [6, 6, 6]^T (where I'm using ^T to mean "transpose to a column vector").
But here, the author means f(1, 2) = 1, i.e. the (1,2) coordinate of the matrix is 1.
f(i, j) := d_i^T * M * d_j
The RHS is using classical matrix multiplication, and the function value will be the matrix' entry at column i, row j.
A matrix represents a linear function taking a vector and returning a vector, written as
w = M v
Matrix multiplication corresponds to function composition.
Vectors can be indexed, and we can view them as a function i -> v[i] defined on the indexing set. We can also define basis vectors b_i, such that b_i[j] is 1 at index j=i and 0 otherwise. Any vector can be written as a weighted sum of basis vectors, with the vector components as coefficients:
v = Σ_i v[i] b_i
where Σ_i represents summation over the index i.
Matrices can be indexed with two indices, and this is closely related to vector indexing: For a matrix M, we have
M[i, j] = (M b_j)[i]
Each column of the matrix represents its output for a certain basis vector as input. By writing a vector as a sum of basis vectors and using linearity, we get the well-known matrix-vector multiplication formula:
(M v)[i] = Σ_j M[i, j] v[j]