Let's start with the author's definition of functional programming, a functional language is a programming language made up of functions, and let's say we agree. Now, what is a function? The author addresses that, too: A function is a concept from math that has been with us for centuries: A function is a static(??), well-defined mapping from input values to output values. The first, and relatively minor, problem here is what one means by "well-defined". Usually, precise definitions are given in a specific formalism, and different formalisms have different definitions of a function. For example, the classical definition of a function is the set-theoretic one, which says that a function is a (usually infinite) set of ordered pairs, i.e., a relation, with the property of being single-valued. This set is sometimes called the graph of the function. Obviously, finite computers can't store such infinite structures, and so they can't represent functions in the classical sense. Functional languages usually define functions as algorithms, allowing to represent a subset of the classical function called the computable, or constructive functions. Those functions also correspond to (some) classical functions, so this is indeed a minor issue.
The problem is that programming languages don't even define constructive functions. For example, the function of the square root is computable, but it (at least its real-valued version) is only defined on the nonnegative reals (let's not worry about representing the reals for now). Another example is the maximum function over a set of naturals, which is only defined for finite sets (and a maximum function over sets of integers is even more complicated). However, most programming languages, even the functional ones, allow their "functions" to be applied to objects outside their domain, something that does not make sense. So functional programming languages call their "functions" partial functions, but partial functions, unlike classical or even constructive functions, are no longer the same well-known mathematical object. So in Haskell, for example, sqrt and max are not functions of the familiar kind, and neither are log, div and many other "mathematical" "functions". Ironically, it is C that treats some functions (such as division) most similarly to math by declaring that division by zero is undefined, i.e., doesn't have to mean anything. Of course, meaningless in math is very different from meaningless in programming, as a program will have some observable behavior even ruled meaningless.
Next, we can restrict our programming language to actual computable mathematical functions, sometimes called in the programming community total functions (to distinguish them from partial functions). Instead of declaring, like in C or in math, that a program that compiles may mean nothing, and instead of specifying a meaningful yet very unmathematical behavior, we can forbid meaningless things (like applying a function to an argument outside its domain) from being accepted as a well-formed program. This is possible, at least in principle, in several ways, one of them (probably not the most convenient) is with dependent types. But doing so is still problematic. For one, if a programming language only allows total functions, the programmer must prove to the language that the argument is indeed in the function's domain. This can be very, very hard (undecidable in general), and part of the reason why no such language is actually used to produce real software (with a couple of minor exceptions that only prove the rule). The other problem is that even then, the objects called functions in the programming language only represent mathematical functions as an approximate denotation, as they really express computations, and computations are not functions -- they are processes that consume time and memory.
To demonstrate the difficulty further, consider that in math, two functions are equal iff their mapping is the same. In those total functional languages this is not necessarily the case, and there are different kinds of equality.
So if you want to be mathematically precise, you can never really program with functions, or at least not the familiar mathematical function. At some point in the definition of "functional" -- whatever definition you use -- you're going to have to make some concessions and approximations. I think that it is well accepted by now that "functional" is a spectrum, and one that includes Lisp.
A short and nice discussion of this can be found in section 1.1 of Jean-Yves Girard's Proofs and Types[1]. I don't think anyone can accuse Girard of not knowing what functions and functional programming are.