NNs do not model the brain, and have basically nothing to do with it. I'd imagine rereading that chapter would be a good idea.
But above all, "pseudoscience" is something different and is certainly not the right word to describe this issue of terminology.
The definition of pseudoscience is "a collection of beliefs or practices mistakenly regarded as being based on scientific method."
Arguably the idea that neural networks reproduce how brains work is pseudoscience.
In the very least, it's a deeply misguided marketing blurb. It's now considered a metaphor, just like the ones driving some corners of the artificial intelligence field.
Moreover, you're dismissing an entire extremely active research field, which tries to understand up to which point artificial and biological NNs are or aren't similar. It's the field, for example, of Turing laureate Yoshua Bengio.
Neural networks CAN reproduce how the brain work (if you're doing computational neuroscience, in which case what you call NNs is something different, and it's an issue of terminology). Even simple binary NNs were originally born to understand the cognitive functions of the brain (McCulloch and Pitts, 1943). The field later diverged (with the advent of backprop), but still today they have a lot in common even inadvertently, for example the representations they learn in navigational tasks (Banino et al. Nature 2018) or in the visual cortex vs. CNNs for computer vision.
The terminology is now fixed and anyone who has more than a passing acquaintance with these things knows that neural networks have almost nothing to do with biological neurons.
either way, the technique has absolutely nothing to do with the biological cells we call neurones -- as much as decision tress have to do with forests.
It is metaphorical mumbojumbo taken up by a credulous press and repeated in research grant proposals by the present generation of young jobbing PhDs soon to be out of a job.
The whole structure is, as it has ever been, is on the verge of a winter brought about by this shysterism. Self-driving cars, due in 2016, are likewise "just around the corner".
ANNs don't yet model the structure of the brain but it seems plausible that they could do in the future as the result of some "convergent evolution".
ANNs have a fair model of individual neurons. Artificial and biological neurons do roughly the same thing when evaluated, but they are connected and trained very differently.
For me it's too much of a coincidence that the two most generally intelligent systems (ANNs and BNNs) are both "linear networks of activation functions".
We have not managed to build general intelligence from any other formalism, and neither has nature.
Viewing ANNs as a poor model of BNNs may be looking at the question backwards. You could say that BNNs are trying desperately hard to model the pure mathematics of ANNs within the confines of biochemistry. The fact that a biological neuron is not exactly a ReLU may say more about the limitations of biology rather than the limitations of ReLUs.
I am unconvinced that "linear network" appropriately defines BNNs. Can you clarify this?
They most certainly do not. The idea that biological neurons are a kind of programmable on/off switch is completely wrong. There is plenty of computation happening inside a single neuron.
The fact that we once thought only the interconnect of neurons is important, and not the individual neurons themselves, is actually pretty strange. We know full well that every single cell is capable of computation, as it must react to its internal and external environment to be able to function. Neurons are more specialized for computation than other cells, and of course the biological neural network is a still a huge part of animal intelligence, but the simple model where we ignored what was going in inside each neuron should have been seen as extremely unlikely to be the full picture from the start.