An interesting thing is that before Dirac, Green was already informally using distributions in the 1830s to solve differential equations with what we now call Green's functions.

Closely related to Green's functions are the fundamental solutions to an elliptic operator, consider the Laplacian and its fundamental solution in Rⁿ (n≥3), what's the Laplacian of the fundamental solution? A Dirac's delta!