MIT D4M: Mathematics of Big Data and Machine Learning [video]

Beautiful explanations of key mathematical concepts such as linearity and associated generalizations. Good to get a sense of how so many real life complex phenomenon can be modeled by combining simple mathematical constructs.

About 25 minutes in he explains that we had the fundamentals in the mid 1950’s. It took half a century for Moore’s Law to solve the problem.

Listened to this talk given on another campus! It's a great intro and makes the concepts feel accessible

His first lecture is good in that he outlines how neural nets are trained to recognize. E.g., can train a network to recognize cats, dogs, and parrots or some such.

IMHO his material on linearity is somewhat interesting but avoids the common approach that what is linear is a function. The usual definition goes, function f is linear provided

f(ax + by) = af(x) + bf(y)

for appropriate cases of a, b, x, y. In linear algebra, commonly f is a matrix, a and b are numbers in a field, usually the real or complex numbers, and x and y are vectors.

Linearity continues quite broadly in math, especially analysis and geometry. One text by G. F. Simmons stated that the two pillars of analysis in math were linearity and continuity.

It appears that after the first lecture, the course is heavily from a paper he and some others wrote on something of a software architecture to ease writing software for the computing for training neural networks.

His architecture is based on his definition of associative arrays which seem to be one, if you will, data structure which can take data from, be regarded as generalizations of, spreadsheets, matrices, maybe usually sparse, some graphs (that is, nodes connected by arcs), etc. For the associative arrays, he defines operations that generalize addition and multiplication of matrix theory, etc. and shows that the operations obey a distributive law that he regards as the key to linearity.

Then the general, infrastructure (interface) software layer is supposed to be small and easy to write.

Some of his examples of anomaly detection from "big data" (it really was big) are impressive.

His framework might significantly help the productivity of building and using neural networks.

First cut, it looks like the main interests in neural networks have been for recognition of speech and images, but his examples likely provide good evidence of other significant application areas.

As might expect from Lincoln Labs, he's fully serious and not just fooling around or pushing hype.

His lecture notes and writing are relatively good.

He has a textbook available at Amazon.

For just the course materials, he has a ZIP file download of 35,802,178 bytes that from just one use of a standard UNZIP command expands into about 640 files/directories (the number of files/directories of his course on my disk is 640 and nearly all of those are from the UNZIP command). Among the many files are some well done PDF files.

He does admit that it is not known why neural networks work as well as they do.