Why focus on outliers and not all quantitative causal relationships in the graph like "tell me what causes X"?
This is why in statistical process control these types of outcome are known as "common-cause variation" and "assignable-cause variation".
I am not a statistician, so I don't know under what circumstances outliers are usually thrown out.
As an industrial statistician, I can tell you: way too often.
Outliers are the signal among the noise. They indicate something. It is nearly always worth finding out what, instead of removing them. If they indicate a flaw with measurement or the process, then fix that flaw and re-do the measurement or re-run the process. Outlier gone! But in a much more informative way.