The book goes beyond statistics and draws attention to "clinical significance" that is crucial to keep in mind when evaluating drugs. Statistical significance helps to inform whether a metric has changed; clinical significance tells whether it actually mattered from a patient's perspective.
Dr. Kirsch's analysis showed that for mild depression treated with antidepressants, there is in fact an improvement in symptoms of depression based on rating scales like the Hamilton Depression Scale - but for mild depression, this improvement in the numbers doesn't necessarily translate into what you correctly identify as clinical significance. However, Dr. Kirsch's own "cutoff" for whether or not such a numerical increase constitutes "clinical significance" or not is itself just an arbitrary number that itself has been criticized.
Also, Kirsch's own metaanalyses (and other metaanalyses) show clearly that for moderate and severe depression (and other conditions), antidepressants are indeed clinically effective (in other words, yes, the benefits actually "mattered" from a patient's perspective), and, importantly, they are highly effective at preventing relapse.
They work, and work well.
In addition, there are a variety of reasons why many short-term clinical trials of antidepressants (which have made up the bulk of clinical trials of SSRIs, for example) are entirely different from the way physicians prescribe antidepressants (and other medications) clinically, and may underestimate their benefit even for cases of mild depression. The beneficial effects of antidepressants increase with longer exposure (i.e. exceeding the length of a six week clinical drug study), probably because of downstream neurotrophic factor(s) increases. Many trials of antidepressants limited dose adjustment or dose adjustment rate. Many clinical trials have excluded patients with severe symptoms or dual diagnoses because inclusion introduces a safety issue or may introduce too many variables. Also, physicians will switch patients to another antidepressant if there is no sign of early efficacy, and while one antidepressant may work very well (probably as a result of genetics), another will not work at all. Most studies focusing on one drug are not going to switch to another medication in the same class (for example, a serotonin reuptake inhibitor) during the trial, even though that's routinely done in clinical practice with good result.
Antidepressants work. Psychotherapy can also be helpful as a sole treatment for mild depression (although please note that psychotherapy has its own set of risks, costs, and drawbacks), and patients often get additional benefit by engaging in psychotherapy along with medication treatment. However, a psychologist or counselor treating a moderately to severely depressed person who fails to refer that patient for evaluation for treatment with an antidepressant would be risking a malpractice suit.