Because that's the modern definition of life expectancy: the average (mean). Probably because the primary driver behind the production of actuarial tables has always been for insurance, annuities, and similar purposes. For calculating costs and valuations the mean is often a better measure because the distribution of deaths isn't normal. Though in modern actuarial tables you'll often find both.
I would assume popular literature quotes expectancy figures from birth because that's the easiest thing to do when you're just trying to give a single simple figure. But actuarial tables have always been just that--tables. Given a person at age X, what's their life expectancy. The first row will be from birth, but people with money on the line have always understood that number is mostly irrelevant. Here's an actuarial table from the 3rd century which was built for calculating annuities: https://en.wikipedia.org/wiki/Ulpian%27s_life_table Interestingly that table seems to use median life expectancy.
Actuarial science was a significant driver in the development of modern statistics and mathematics, especially as the market for financial products exploded over the past several centuries. The tables and models for life expectancy from 150 years ago were startlingly accurate, even in their forecasts of changing life expectancies. When Social Security was passed in the 1930s the models for how life expectancies would change, both from infancy and across all the age brackets, were spot on with actual changes up to the 1990s.
Even then though, in e.g. Victorian times, the median (50th percentile) gets skewed by horrendously high infant mortality.
Looking at the 75th or 80th percentile, or the life expectancy for an 18 year old, is a better measure. High scores for the 75th can mask deaths due to inequality to some extent, but because wealth distributions are _so_ skewed at the top, the 75th percentile typically looks more like an average/poor person than a rich person.
It's this context that I think explains things like infant 'exposure' in various cultures over time that we can't even imagine today - parents simply leaving their child out to the wilds, essentially to die - while retaining that cognitive dissonance that perhaps he might be found and raised by somebody, as was a frequent component in the hero legends, such as with Theseus and Perseus.
[1] - https://ourworldindata.org/child-mortality-in-the-past