> Regression is any algorithm that takes a collection of inputs and predicts an output.
I get that the author is writing for a certain audience, but this is a gross over-simplification.
I get that the author is writing for a certain audience, but this is a gross over-simplification.
You may say it's only regression if it fits your favorite framework, like giving confidence values or goodness of fit etc, but that's not true in general. There so many variants, like Bayesian probabilistic regression, nonparametrics, neural nets, random forests etc.
Either use is ok, but e.g. a tree model predicting if an animal is a dog or a cat is not regression by any definition.
You're right that regression is a type of task, not a type of solution. The conceptual difference is important to understand if someone does not yet know it. But it's not an all or nothing set-in-stone thing. It implies there is a waterfall design to these ideas, as if some oracle posed these tasks to us and then we started finding solutions to each of them separately. But actually in many cases one and the same algorithm with small tweaks can tackle multiple tasks. Sometimes we have the hammer (algorithm) first and then the nails (task).
How you build your taxonomy and how you categorize one approach or another is not the same as learning and understanding. I always had an issue with this at university, where some lecturers would confuse learning lists like "what are the 3 areas of field X" or what are "the 4 principles of approach Y" etc.
The world is not structured according to subjects, fields, subfields etc. It's not a single hierarchy, but a big mess of similarities, like a graph or a multidimensional space. The map is not the territory and so on.
Terminology is necessary of course for communication and structuring books etc, but I like to see it merely as a utilitarian thing. The categories provide a scaffolding so that learning can happen. Studying the vocabulary and various outlines and nested hierarchies is a useful part of the journey but should not be confused with actually learning the thing itself. You could in principle learn all about regression without ever learning the word "regression".
The name actually originates from statistics, named after the "regression towards the mean" phenomenon, where they observed how the adult height of people is closer to the average compared to the height of their parents. So a tall person will have tall children but less extremely tall (in tendency, some will of course be even taller). So confusingly, regression literally means "going back".
Statistics :)
As a subfield in computer science, of course the concern has often been on algorithmic complexity and similar. But that is nascency exposed, in my view, and likely not representative of a fully mature field.
Armchair thought (not a historian of economics): I think Econometrics followed (and continues to follow) a similar evolution -- start with the goals (identification of model parameters, identifiability, KPIs), improve statistical validity and relevance, annotate dead ends or less common routes, continue on trucking.