Counterfactuals are not causality
michaelnygard.com
michaelnygard.com
"Why are the lights off?"
"Because I didn't turn them on."
That's not a counterfactual, it's a fact. Just like the admins not configuring file purging is a (presumed) fact in the scenario under discussion in the article. Negative statements are not counterfactuals. Now, they may not be helpful for a 5 Whys analysis, but that's a separate thing.
Counterfactual: If the admins had turned on file purging, then the volume would not have been full.
Why? Because they didn't turn it on so this is a counterfactual in that it is dealing with facts not in existence in the reality under examination and reaching a conclusion of how things would have been different with those different facts. But nearly all the causes in a 5 Whys can be seen as counterfactuals if phrased correctly:
"Builds did not complete." Why? "Kubernetes could not start the pod, and the operation timed out after 1 hour."
As a counterfactual: If Kurbernetes could have started the pod, then the operation would not have timed out after 1 hour and the builds would have completed.
"why did the disk fill up" would be most directly explained by the most recent write. That's not useful though. It doesn't really matter what the last write was.
In order to get the "because" we have to imagine another state of the world where you DID turn on the lights, and further speculate that this would have been effective to illuminate the room. That's counterfactual, because the fact is that it's dark in here.
It seems like it would be strange to reject as an answer to "why do blind people tend spend less on lighting?" with "Because they don't tend to use the lighting, because the lighting isn't useful to them, so they don't turn the lights on as much (unless someone else who isn't blind is there).".
How else would you answer that question? (modulo little wording changes and such)
But no, in a meaningful sense it's not a true statement about the world. It's a good guess, and for a statement that simple, there might be no need to go further. For a statement about why a volume is full with too many files, the point is that a nice-sounding "if only" isn't something you can test or be certain of.
Your answer about blind people is a very good guess as well, and the same one I would make, but it's not data.
A counterfactual requires posing a situation with changed facts. I stand by my statement, saying "X because not Y" is not a counterfactual on its own. "not Y" is still a (potentially) factual statement. And guesses still aren't counterfactuals. "Maybe the lights aren't on because we didn't flip the switch?" That's a hypothesis, and a testable one. Still not a counterfactual. If we know the flip hasn't been switched, then it's just a fact even if the statement is in the form of not X.
Now consider the opposite. The lights are off because the bulb burnt out. That's a lot less ambiguous. If you want to doubt the causality you've got to get all mystical.
If me turning on the light is a necessary condition for the light being on, then it implies that if I did not turn on the light the light is off. There is no speculation. This is also what distinguishes this one "counterfactual" from all the others.
The article seems to completely ignore that we can readily argue about negative statements. I highly doubt that airplane crash investigators would avoid counterfactuals.
What they say is that counterfactuals stop the search for the root cause. Not monitoring disk space isn't a root cause of the filled disk. The root cause might be "application has suddenly written too many logs". This gives another set of counterfactuals, like "if your application was configured properly, it would write less logs". But it also gives an option of continuing the search for the root cause, which might be, for example, "there is a bug in the application which triggered the log writes".
I think the point of methods like 5 whys is not to stop at the first set of counterfactuals and deepen the search.
Had the landing gear been retracted in your hypothetical world, then it would be a counterfactual, but it probably wouldn't arise because it's not something out of ordinary - the landing gear retracting is a usual procedure. I think humans are more prone to make the mistake that is described in the article (stopping to search for the root cause) when they see a counterfactual conditional which premise is, in the factual, a procedural error (for example, "operator failed to monitor the disk space"). We see an error and mistake it for the root cause.
By arguing there are no necessary conditions you are essentially saying we can't use mathematics to describe nature and there are no laws of physics.
For it to fall from the tree? Well, that's a linguistic trick; for it to fall from the tree it must have been hanging on the tree, so that necessary condition is actually a tautology.
Physics describes a world with statistical regularities, not absolute laws. Even a clockwork Newtonian universe is not time-reversible; it has multiple pasts that can lead to the observed present.
E. g. "The apple hit Newton on the head" could be explained by:
1. The apple was hanging on the tree at `t - 1` and the wind shook it loose.
2. The apple was flying through the air because his mother threw it and it collided with his head
3. The apple spontaneously generated due to as-yet-not-understood quantum fluctuations 3 meters above his head.
Something to be a necessary condition has nothing to do with time-reversibility, it simply states a logical relationship
Let me give you another example: To accelerate a body with mass m it is necessary to exert a force onto it. All mathematical descriptions In fact equivalency, requires conditions to be necessary and sufficient.
Your argument that physics describes a world with statistical regularities is a maybe a nice philosophy, however most physicists (and regular people) do not believe we live in a simulation, but physical laws describe the real world.
To be pedantic, it’s not quite that either. The logic theorem states if A => B then !B => !A. The => means that A is sufficient for B, because “false => true” evaluates to “true” but “true => false” evaluates to “false”. If it were “necessary”, those statements would evaluate the opposite way.
This means that if me turning on the lights is sufficient for the light being on, then the light being off is sufficient to deduce that I did not turn it on.
You are simply stating that me turning on the light is not a necessary condition, but a sufficient condition (which is a reasonable argument).
My argument was that if the statement "if the light is on it was turned on by me" is true (i.e. the me turning on is a necessary condition), then if I did not turn on the light the light is off [edited after mistyping before].
I think this works better with other examples, because as you state, we typically think more of the turning on as a sufficient condition.
Yes, it wasn’t my intent to contradict your overall point, just to correct your logic.
My argument was that if the statement "if the light is on it was turned on by me" is true (i.e. the me turning on is a necessary condition) then it follows that if the light is off, I have not turned it on.
I disagree with you here. The counterfactual, or at least the logic theorem, for the statement [light is on] => [I turned the light on] (me turning it on is necessary for it to be on) is [I did not turn the light on] => [the light is not on]
That’s sound, though it’s not a particularly insightful leap of logic.
If you translate this to the author’s point, they’re basically saying that !A => !B doesn’t tell you that A => B. And that’s correct as far as Boolean logic goes. I am unclear on whether we disagree on that point.
>I disagree with you here. The counterfactual, or at least the logic theorem, for the statement [light is on] => [I turned the light on] (me turning it on is necessary for it to be on) is [I did not turn the light on] => [the light is not on]
Yes you're correct and that's what I corrected my statement to where you managed your reply in between.
However my original statement was:
>If me turning on the light is a necessary condition for the light being on, then it implies that if I did not turn on the light the light is off.
so A=>B therefore !B=>!A. Where A == "light is on", and B == "me turning light on". I agree that it's not particularly insightful.
My point was regarding the statement "The whole point of the article is that you can't get causality from things that didn't happen" from the OP I replied to. That's just not true, we can make statements about causality for things that didn't happen, because we can deduce them from logical conditions.
>If you translate this to the author’s point, they’re basically saying that !A => !B doesn’t tell you that A => B. And that’s correct as far as Boolean logic goes. I am unclear on whether we disagree on that point.
Although we agree in principle I disagree what you say is the authors point.
From the article: >They [counterfactuals] express wishful thinking about an alternate history where the bad event didn’t happen. Because they represent “events that didn’t occur” they cannot have caused anything.
This is a statement I disagree with. To me that sounds that the author essentially says we can not make statements of the type of !A => B (or !B). To me that is wrong and also not a counterfactual. Counterfactuals, are statements about a hypothetical event, i.e. "If I had not turned the light on it would be off now". To me though the author seems to extend the definition of counterfactuals to mean "events that did not happen", maybe he means the correct thing, but both his words and examples do not reflect the correct meaning IMO
Wikipedia doesn't seem to agree with you:
"The term "Counterfactual" is defined by the Merriam-Webster Dictionary as contrary to the facts.[2] A counterfactual thought occurs when a person modifies a factual prior event and then assesses the consequences of that change.[3] A person may imagine how an outcome could have turned out differently, if the antecedents that led to that event were different. For example, a person may reflect upon how a car accident could have turned out by imagining how some of the factors could have been different, for example, If only I hadn't been speeding...."
The forms the article mentions are like those "if" statements.
>That's not a counterfactual, it's a fact.
That's because there's a one-way casual relationship there. Not the case with TFA examples.
1. https://plato.stanford.edu/entries/causation-counterfactual/
The main idea: Let Y be the set {y, not y} and X be the set {x, not x}. y denotes timing out, and x denotes “Kubernetes not starting the pod”. We have some distribution P(Y=y|X=x) (in this case, the probability of “timing out” (y) given “Kubernetes not starting the pod” x. The counter factual distribution is NOT the same as P(Y= not y | X= not x) (the “probability of not timing out given Kubernetes starting the pod”). The counterfactual is P(Y’=not y| X=x, Y=y, X’=not x), or “the probability of not timing out given that it did originally time out when Kubernetes did not start the pod, and given that this time it did start the pod”.
This aligns roughly with what you are saying above.
Also, "the car didn't run because there wasn't enough gas" is fundamentally different from "the car didn't run because the engine broke down".
"I'm tired because - I stayed up all night playing games", or, "I'm tired - because my child screamed all night long," are more along the lines of what the post suggests are useful, and I can see how they're fundamentally more useful to getting at the root cause than the counterfactual, "I'm tired because I didn't sleep."
But this example is very simple and makes the argument less important I'd say.
The author of TFA seems to suggest that explanations where Y is "not Z" are inherently less useful (I don't think so) and also calls them "counterfactual" even when "not Z" is a fact, which is confusing.
No - the definition the article refers to is this: adjective - relating to or expressing what has not happened or is not the case.
I used "because" to illustrate the "looking for root cause" use case, like what the article was referring to. Regardless, that doesn't make it counterfactual per the definition the author is using.
To me the part that makes the counterfactual less useful when determining root causes is the human element the author mentions - looking for root cause tends to stop once they're brought up (they're the end of the causal chain in discussion), and blame starts to get assigned. I've seen that happen, and it hasn't helped resolve the issue, so avoiding that seems useful. I don't see a problem with using a counterfactual statement if it really is the root cause and I'm blaming myself for something.
Well, they are different because they propose different causes. Either of these statements (and the so-called counterfactuals in the article) might be true, depending on the actual cause.
The distinction that matters here is that statements of causes are not solutions, and solutions are elicited by different questions than are causes ("what can we do about it?", rather than "why did it happen?") It does not really have anything to do with counterfactuals vs. causes, and the author's actual point seems to be about how to present solutions.
Which... May not be true.
https://plato.stanford.edu/entries/causation-counterfactual/
Its still a somewhat live debate in contemporary philosophy.
The medicine saved the patient's life is true if the patient would have died had the medicine not been administered. How do we determine the truth of such a counterfactual?
In practice we do large scale randomised trails to attempt to simulate a counterfactual to test the causal powers of the the medince - but its probably beyond our powers to know for certain that the counterfactual is true in the singular case.
Philosophers are interested in what makes a counterfactual truth or false, and whether understanding a counterfactual presupposes causation or whether a causation can be understood in terms of counterfactuals. Nobody doubts that counterfactuals and causation are releated, and few doubt that counterfactuals can be reduced to their logical or epistemic counsins.
Implicitly there's this idea that every counterfactual can be contained and localized, but that runs counter to my understanding. Change one thing and you change the entire universe. That's true even classically -- even before you bring in quantum physics and its problems with counterfactuals.
Perhaps counterfactuals are a good model of the psychology of causation. And in fact, I suspect that psychology is as deep as causation runs; it doesn't in fact have a direct physical or epistemic analogue. All of which is fine and important, but I feel like it diminishes the value of the effort put into it. It's not studying VLTIMATE TRVTH, just a quirk of how brains think about stuff. And that would mean you need to use a very different toolset to work on it.
- Build didn't happen, why?
- Pod didn't start, why?
- Disk was full, why?
At this point if counterfactuals are off the table, the answers become tautologies. Although if everyone is in agreement that counterfactuals are not helpful, this could become a powerful heuristic in determining that you've hit root cause and can shift from searching for root cause to designing mitigations for the root cause.
I guess that's the point of the post, but it seemed weird to start a scenario saying "it goes off the rails here, because of XYZ" and not come back and give the better version of that scenario.
- A process wrote 125 GB of logs, why?
- It was compiled with VERBOSE=true, why?
- The build config was changed on 7/12/2021 by Steve, why?
- Issue #5988, need to collect data one why that process crashes so much.
Key points from a good article.
Definition: "A counterfactual is a statement about how the world might be different now if something had happened differently in the past. It’s a kind of “alternate history” idea."
"Here’s the rub: a counterfactual cannot be a cause. By definition the counterfactual did not happen, therefore it cannot have caused anything. Only events that actually occur can be causes of other events. Causality should be stated in a form “Because X then Y”. The statement “If not X then not Y” is not an explanation, it is a kind of wishful thinking about how the past might have unfolded differently."
"When performing Five Whys it is important to avoid this counterfactual leap. Stick to the events that actually occurred. [..]"
"Try to reformulate the counterfactual as a statement about future prevention:"
1. If we configure file purging, then this won’t happen again
2. If we monitor for “volume full” conditions, then this won’t happen again
3. If we clean up files from old builds, then this won’t happen again.
These are useful statements. When formulated this way, they’re clearly talking about the future and not hypothesizing an alternate history. "
For example "The admin did not configure file purging." isn't a counterfactual unless admins were in fact purging files.
A better phrase might be subverted expectations, along with the admonition to make expectations more explicit with the author's if-then examples. Inaccurate expectations are just as important to analyze and fix as hardware and software bugs.
Counterfactuals are actually very powerful and useful because they do let us analyze the past instead of just the future. I have found that chasing counterfactuals usually bottoms out in known failure modes in practice. For example tracing back that lack of disk space in build nodes would probably arrive at either "insufficient design" or "insufficient review" or even "insufficient understanding", all of which are pretty well-known and common engineering failure modes and we already know the solutions in the future; spend more time on design, review with more people, ensure education is happening. It's actually quite useful to do this because as a trend over many postmortems "insufficient design" has very different organizational solutions than "insufficient understanding" and it's a meta-level counterfactual of "if we were better at (design|understanding) what problems would we avoid?"
Actually, he lost me earlier, cos "The admin didn't configure file purging" is something that did happen: the admin failed to take an action. The event happened even if the action didn't; the article confuses the two
It doesn't really matter tho since this article is really just a piece of advice on how to handle incident reports in a tactful and constructive manner, not a serious argument about causality
___
If there is a domino, and I push it, and it falls over, I think most will agree that "It fell over because I pushed it.".
If I don't push it, this article seems to say that "It didn't fall over because I didn't push it." is not a causal explanation.
And, maybe that's right, but I don't think the "because 'if I had pushed it, it would have fallen down' is a counterfactual" is convincing reason for that.
Because in the case where "I don't not push it" and it "doesn't stay up", shouldn't "It didn't stay up because I didn't not push it" be a valid causal explanation, being equivalent to "It fell down because I pushed it", yet, in this case, "if I hadn't pushed it, it wouldn't have fallen down" would be a true counterfactual?
-- oh, this article is about advice for language use in interpersonal stuff (especially in a work environment), and maybe practical problem solving in general, not studying metaphyics-or-whatever of causation, ok.
She discusses some views on how to reason about common sense statements like "Smith's failure to water her office plants caused their death". The salient views covered are:
1. causation by absence is impossible (making the plants statement false)
2. causation by absence is possible and attributing an absence to a cause is sound when the absence is "abnormal". This could be because Smith normally waters the plants, it was Smith's job to water the plants, or Smith made a promise to water the plants.
3. causation by absence is not possible, but it is valid to offer "causal explanations". This is the suggested approach in the paper. A causal explanation provides information about the causal history of an event, but doesn't need to specify a specific node in the causal graph. C causes E is a causal statement. E because C is a "causal explanation". E because C doesn't necessarily imply C causes E. So adopting this approach, The plants died because Smith failed to water them is a true statement that provides some additional explanatory information, Smith failing to water the plants caused their death is not.
I think these distinctions may have some minor implications on how you perceive blame.
1. The point of counterfactuals is not to "prove" causality; it's to help you detect possible causal routes, in the presence of given events under examination. E.g. if you know events X, Y and Z co-occurred, and you want to know if X or Y could have possibly caused Z to happen, the counterfactual formulation can answer that question. If not X implies not Z, whereas Y does not alter Z, then you have eliminated Y as a causal agent, and have identified X as a possible causal agent (in fact the only possible causal agent, if Z could not be influenced by another external, unidentified agent, or by itself).
Furthermore, in the presence of a number of counterfactual statements, you can allocate responsibility and blame, linked to expectations of events. So aliens is a silly example, unless for whatever reason the author has a very high prior expectation for their existence, which exceeds that of file purging errors.
Many times I dont have visibility into the entire system so the best I can give the client a good set of counterfactuals on the bad policy they used to get to that place.
I.e. I administer a pill with an active drug to a treatment cohort. What would have happened had I administered no drug (sugar pill) to the same treatment cohort.
Counterfactuals require a causal model. More so, they belong to at least one causal model. This can be a mental model, a formal one, a precise one or a diffuse one. Nevertheless, counterfactuals "come" with an implied model. People may disagree about which model they talk about, and which model is correct. Such a model may or may not be "precise" enough to identify useful parameters, which is the real rub here.
For example, the three "Counterfactuals" the author presents presume a model where purging the cache (etc.) directly causes the problem to go away. Each of those statements might come from the same or from a different causal model.
The problem, which I feel make this approach not useful, is that the author has no model in mind. In other words, they switch between different models. One counterfactual is, for instance, "If we don't live on earth, then we don't care about the cluster". Let's ignore that the outcome variable is different. The model with "We are not even alive" as a variable is not a reasonable one. First, it is empirically wrong. Second, it does not "identify" parameters (relationships) of interest. These two points are what we strive for when using a model (which we do when we construct counterfactual, even if it is a verbal model!)
The author posits that one can add unlimited counterfactuals. What the author means is that one can add unlimited variables and relationships (however diffuse) to the model. This is true.
However, we still need to judge whether the model is good, whether it is helpful. In drawing such analogies to statistics, one should not forget the other concepts.
From a statistical standpoint, the author begins with a model that does not identify causal parameters associated with the failure to start the cluster. They then add "counterfactuals", that is, they expand their mental model to new causes. They find that any of these variables may cause a counterfactual where the cluster starts.
In my view, the author is only half right. Counterfactuals, as presented here, imply a causal model (perhaps not a precise one). Counterfactuals can be useful, if the underlying model is useful. Whether this is the case, depends largely on factors the author does not discuss in the article.
Edit: It may also be helpful to distinguish a counterfactual from "observations with exogenous variation". For example, if we purged the cache some time ago purely by accident, and the cluster did start, we might take this as a hint that purging the cache allows the cluster to run. This is not a counterfactual. It can be used to identify causal chains. If we were sure that nothing else was different (in a certain statistical sense), then we could "identify" a causal path from cache purging to staring the cluster. However, to determine that, we again need a model. In fact, we need a model and assumptions about the observation (what is called data generating process) jointly - because really, they are the same thing. For example, if the cache was purged because an engineer discovered an error somewhere, and then did some other action Y - perhaps it was not the purging of the cache that allowed the cluster to run. It may have been action Y. Here, purging the cache was not "exogenous" enough for us to infer the correct reason.
Identification of a model/DPG is a deep concept, but a crucial one to finding cause and effect.
Not only are counterfactuals infinite in number but ALL hypothesis are infinite in number. You can literally say ANYTHING about why something didn't build. I can say someone didn't purge some files or I can say someone purged files or I can say aliens destroyed the server with a laser. There are an infinite number of statements that can be made here and whether it's a "counterfactual" is irrelevant.
What the author doesn't know is that NOTHING can establish causation except for a very specific type of experiment.
Basically there is nothing that can establish causality UNLESS you have control over the causal switch. You flick the switch randomly and if the randomness of the switch flicking correlates with the consequence you have then have established causation. BUT you have only established it to a certain correlative degree. It is fundamentally impossible in this reality to establish causation to be 100%. True causation can never be fully verified and at best the experiment I described is the best fuzzy verification. If you truly study how science works you will find that this flaw extends to all of science and therefore reality. In reality as we know it, nothing can be proven. Proof is fundamentally impossible. If you understand this concept then you will truly understand what science is.
Anyway, Going to the authors Kubernetes example, his "counterfactuals" are basically null hypothesis and can be helpful in establishing causation but by themselves are pointless.
It's simple... If you hypothesize that something didn't build because the admins did not "configure file purging" then to establish causation is SIMPLE. Configure the file purging and see if your system builds. Then remove the configuration and see if the system stops building. Do this enough times and you have established causation or zero causation to a certain degree.
That's it. This is science and probability and statistics. So crazy to see someone mention causality without even knowing what it really is.
Causation is "established" in the usual sense by having a causal model of a data generating process. Such model may or may not identify causal parameters given the data. That is, it is not the "randomized control trial" itself that establishes causation, rather it is a more general concept of statistical identification.
Case in point, there are instances where a RCT fails. For example, even if you can assign treatments, you might find that treatment and control groups influence each other or whatever else can go wrong in a RCT.
By contrast, there are instances where we do not control the causal switch, but we can observe "natural experiments" nonetheless. Usually, causation is fuzzy, since there are at least some assumptions which need to be made for us to believe the analysis to be causal. However, really, the same is true for an RCT.
For example, if you observe an engineer sometimes purging the cache, and sometimes not purging the cache, you do not actually control the assignment of treatment. However, it is easy to think of a mental model where observations of this engineer, and the state of the cluster, gives you high confidence to proclaim: "Purging the cache makes the cluster start!"
So, this is just to expand on what you wrote: Causation is not a precise concept, because it depends on our model of the DGP and the data we actually have. Identification of causal parameters depend on these factors, and thereby on assumptions - this is true for RCTs where I control the treatment, just as much for observational studies where I do not. Even if I assign treatments, I have to make the assumption that my assignment is truly random, and that there is no non-random factors influencing my observations (etc).
In quite a broad sense, causation as a scientific concept depends on assumptions I need to make. If I get to control the treatment variable (or variable of interest), it is typically easier to create "exogenous" variation that identifies causal parameters... in English: Allows me to establish causation.
However, the whole concept does not depend on controlling or not controlling treatment. RCTs can fail, or there are instances in which I can establish and measure causation even without controlling the treatment.
It may be that I'm coming from a more philosophical perspective. What is randomness?
If randomness involves free will then only an entity with free will can create that random variable aka only another person or yourself can create randomness.
Pedant-ism aside, can you think of a "truly random" data generative process that doesn't involve someone with their hand on the switch?
I guess this is the assumption I am making in my explanation: Only people with this magical thing called "free will" can generate random data, nothing else can be trusted to be truly random so you or another person has to have their hand on the switch in order to establish causality.
Note that even your example stayed within the limits of this assumption. You were possibly unable to choose another example that didn't involve someone with free will. Either I have my hand on a switch, or I'm observing another engineer who has his hand on the switch.
Typically to illustrate a point one would usually pick an outlandish example that strays very far from a human having his hand on a switch and show that the example is STILL true. But you didn't do this and that may mean something.
You see, in the pragmatist view of a statistician (and in the math that goes with it), randomness - for example in the form of statistical independence - is not required to identify causality (or causal parameters of a model).
Instead, we need the "switch" variable to be uncorrelated with things we do not observe, or can not adequately model. This requirement holds for a random experiment as much as for an observational study. However, you will quickly convince yourself that this is an assumption. It depends on the data and how we got it, it depends on what we assume about the factors at play, and it depends also on the model and how we take into account factors that influence the outcome.
Specifically, the identifying assumptions for a causal model are untestable. Let me emphasize this again: Every causal analysis is fuzzy, because they are only causal based on the assumptions.
As such, the question of true randomness is somewhat ancillary: Statisticians know that the idea of causality is by default not a philosophically "true" concept. This, of course, leads to a pragmatist view.
One last point: Between free will and flipping a switch, and a causal analysis, stand many assumptions about the data, the measurement, and so forth. In that sense, statisticians probably do not feel that a random flip switching is less assumptive than any other causal analysis.
Related to this, let me offer you the perspective of social scientists. They feel that human behavior IS influenced by other factors, just as a machine or physical system would be (broadly speaking). That is, they would reject the notion that free will is indeed (or at least always) random. In such studies, for example, choices from individuals would also be subject to intense scrutiny with respect to unobserved influences or other factors that might endanger the exogeneity of the observed choices.
Causal identification has been likened to a Zoo, with many kinds of animals in them. I am sure the philosophical idea of causation is part of the Zoo. On the other hand, statisticians speaking of causal analysis find themselves unable to "prove" any sort of causality, even weak forms as indicated above, without underlying assumptions. Case in point, a randomized controlled trial is chock-full of identification and exclusion assumptions. They are just standardized and accepted.
Sorry again that I have to rely on the statistical view - it is the only one I know a little bit about.
I feel the entire field of philosophy is mostly useless and full of categorization errors. I don't buy into it, I just use the word for speculative stuff. A lot of the things that the actual field of philosophy chooses to speculate on (Animism for example) is just random garbage. They also pair religion and logic together as two peered pillars of study, as if religion is just as fundamental as logic. I have little respect for the entire field from a formal perspective. In my reply, I was using the word informally.
>You see, in the pragmatist view of a statistician (and in the math that goes with it), randomness - for example in the form of statistical independence
No we're talking about randomness as a fundamental phenomena in a procedural and physical universe. Meaning that what is a definition of a pure stateless function that takes no input and will always output a completely random number? The probabilistic definition of a random variable does not actually produce such a function and sort of avoids the concept of randomness all together.
In physics, quantum physics specifically, such a function is not defined but is taken as an axiom. The actual location of a atom after decoherence is assumed to be axiomatically random.
>Instead, we need the "switch" variable to be uncorrelated with things we do not observe, or can not adequately model.
Arguably everything in the universe is correlated and shares a causal connection. A butterfly flapping it's wings in China causes a hurricane on the other side of the world. On a grander scale: A perturbation of an atom in one corner of the universe eventually changes the parameters of every atom in the universe.
According to certain theories in physics information can only travel as fast as the speed of light. Therefore according to this logic all things can be causally connected but only up to a point. If you imagine multitudes of infinite spheres continuously growing out of an atom at the speed of light, then everything outside of a sphere that started growing at t0 is guaranteed to not have a causal connection to any events that happened to the atom at t0 or after.
However in math we create our own playground devoid of all the messiness of the universe and it's atoms and limits to causal transmission across large distances. Within this world there seems to me, to be an issue with your notion of a switch to be "uncorrelated."
For something to share a causal connection with another thing, both have to be correlated. You cannot establish causality if the switch is uncorrelated with the observation. The result will always be a failure in establishing causation if you are selecting this type of switch.
There is a paradoxical problem with the definition here. I get your intuition and you understand mine as well but this definition seems to be incomplete or not an accurate illustration of both of our intuitions of this concept.
Using "randomness" as a concept and leaving "correlation" out of the definition seems to better fit our intuition of what's going on when *I* or another human flips the switch. After all, you cannot have a causal relationship without a correlative one.
>Statisticians know that the idea of causality is by default not a philosophically "true" concept.
I would say the just "don't go there." They stop at the definition of a random variable and don't dig any deeper. https://en.wikipedia.org/wiki/Random_variable <--- on wikipedia a random variable is defined informally in terms of "random phenomena." Additionally the formal definition is defined in terms of "possible outcomes" so they're really side stepping the problem here. But that's ok, it's not really needed to play the mathematical game of probability.
>Related to this, let me offer you the perspective of social scientists.
A social scientists opinion is not any more valid than an average persons opinion. A social scientists views human behavior as a black box. They don't just put one human brain in a black box, they actually put a population of human brains in a black box and they try to analyze everything from that perspective. It simplifies things but inevitably there's a lot of missing information. One piece of this information is whether the output of the black box is random. If you can't peer into the black box you can never know if it's random or not random. Thus the social scientists viewpoint doesn't mean any more than someone who isn't a social scientists because most people view the brain as a black box as well.
That being said if there was a field that could closest determine "free will" it would be neuroscience and quantum physics. When I am flicking that switch and attempting to randomize the timing of it all what neurons are firing and what chemical reactions are occurring in my brain? What is the exact mechanism producing this action? Does the randomness of the quantum world leak out of a chemical reaction to influence the macro output of my timing on flipping that switch?
>That is, they would reject the notion that free will is indeed (or at least always) random.
Again they can't reject anything or accept anything related to this. To them, this part of the universe is underneath a black box.
>Sorry again that I have to rely on the statistical view - it is the only one I know a little bit about.
Who says you have to restrict yourself to viewpoints from certain groups? When you restrict yourself to "the statisticians" viewpoint or "the social scientists" viewpoint you are also restricting your intelligence.
However, the system is indeed consistent. And so, in this framework, neither true randomness (which is not a concept that has a definition here) nor independence is mathematically required for causal identification, instead, the switch variable needs to be uncorrelated to unobservables (and so on). That it needs to be correlated to the target variable is obvious, as you say. That is all tautological and simple to show.
I should also note that both independence and correlation are defined in terms of statistical properties that can hold irrespective of whether the world has true randomness. In a way, think if statistics as tying to the world that we see (the data) a consistent way of understanding and inferring. Now, whether the world actually works in terms of measures and moments, well, that is another question. And, I would propose, one that will be difficult to decide, in that it transcends our best ability to work with observational facts.
Note that there are other conceptions of causality and inference, distinct from statistics. I have worked with a group using models without any randomness, relying instead on the modeling of fully deterministic chaotic systems. They can indeed also fit the data, even if they are less practically useful.
As it turns out, a procedure of identifying causality, getting useful results, looks not too different.
So yeah, I think we agree in basic terms. In your view, causality as it is typically applied is fuzzy since its axiomatic framework does not arise from physical, philosophical or perhaps metaphysical facts as you believe or know them to be true. In my view, causality is fuzzy because any pragmatic definition, including one based on experiments, is based on assumptions without which causal identification is impossible. For instance, no matter what experiment you construct, we will need to assume that the operator switching randomly has not been paid money to choose a predetermined sequence by some James Bond villain. Of course, only one of us can be the operator, and so any pragmatic definition of causality (in being communicable) will eventually be fuzzy.
What is an unobservable? Do you have a source for the formal definition of causal identification?
In wikipedia, they immediately skip logics and math and jump to science which indicates to me if a formal definition exists, it's controversial and likely not well known.
To identify a causal effect of x on y, you need to exclude the possibility that x is correlated with z and z influences y. If it is so, you need to model z in some way (here, it depends on what we talk about). There are also other considerations, but that is perhaps the most pressing and obvious one. You can quickly convince yourself that the above must hold if a causal effect is to be identified. Under assumptions, for example induced by experimental conditions, the above is also sufficient to identify a causal effect.
The absence of any unmodeled z ( eg the Bond villain in our aforementioned randomness experiment) is essentially always an assumption and not a provable fact, which motivates my earlier discussion. Note also that measuring, modeling or otherwise taking into account any such Z depends on the Z and therefore the data generating process (aka the situation at hand )
Given that causal identification requires assumptions on the DGP, no general or non model based definition can exist. That is, we CAN say what we mean by causality, however the task of identifying causality has to be stated in terms of a of the data generating process.
Perhaps herein lies the confusion.
Based on the DGP, mathematical definitions of causality do of course exist. For example, in the framework of switch and outcome, as defined, causality is semiparametrically identified if the switching is uncorrelated with other factors that are not modeled. In that case, we identify the qualitative direction of causality, or causal effect. For example, if the switching is true randomness, as per your definition, then the condition is fulfilled. However, randomness is only sufficient not necessary.
What a causal effect is, well, that means depends on the data and situation, for example what values outcome can take and what measures you apply: Expectations, Distributions, Order Statistics and so on. If we see three people operating switches in terms of dials, we would for example be able to identify the average marginal effect of dialing on outcome. This would be an estimated expectation.
Formal definitions of these concepts can be found in, hopefully, any modern statistics textbook.
Otherwise, there are many formal treatments available. For the concept of statistical identification, Lewbells article „Identification Zoo“ is a good introduction. Formal definitions of identification and causal identification (albeit written for econometricians) are presented. http://fmwww.bc.edu/EC-P/wp957.pdf
The first sections give a good understanding why it only makes sense to talk about causal effects and causal identification, if we have a model (restrictions on the DGP) in mind. While the text is generally about statistical identification, you can do the following: Assume that based on your DGP/model, theta_0 (in the text) is the parameter telling you, whether X causes Y. Then, reading section 3 is probably sufficient to get an understanding of what causal identification is.
As it applies to causality and counterfactuals specifically, read Pearl as one framework, and research on Rubin for another. There are now, I think, good books on amazon treating both in more generality, especially outside of econometrics (which is or was for some time leading in non experimental causal analysis). Then, literature on experiments is also concerned with causal analysis, though since the distinction between correlation and causation is less of an issue, the literature has less focus on stating what exactly makes a procedure a causal analysis and what does not.
For that reason I actually recommend looking at the non experimental literature first, since the need for definition arises.