First-Class Automatic Differentiation in Swift

Julia relies heavily on type inference, similar to all variables to be marked auto or auto& by default in C++.

Functions are templated on the argument types by default, in C++ terms. So if you call a function f(x) first with an integer and then with a floating point number, Julia will generate separate machine code optimized for each case. Annotating function arguments with types only serves as a filter for whether the function applies to certain types, and can be used for method overloading (function template specialization in C++). The fact that functions are generic by default lends itself well to forward-mode automatic differentiation using method overloading.

Julia is still a dynamic language though. This means that if a variable is bound to an integer in one branch of an if-statement while a string is assigned to it in another branch depending on the value (not type) of a function input, then the variable's type cannot be inferred and needs to be 'boxed', i.e. the runtime type needs to be stored along with the data. In such cases, function dispatch can no longer be done at compile time, and needs to be done dynamically instead. While this is useful for rapid prototyping, it does result in reduced performance, so it should be avoided in code with high performance requirements. The type elision part of this is somewhat similar to boost::any.

That's an...interesting statement, considering that neural networks are nearly as old as software itself. The original Pitts-McCulloch paper was published in 1943. Active use of them has certainly been ongoing longer than that blog author has been alive.