From Asking Causal Questions to Making Causal Inference

From Asking Causal Questions to Making Causal Inference

The warning – “correlation does not imply causation!” – has probably been drilled in your introductory statistics course to stop you from making reckless causal statements. If you are not careful, you may believe that more ice cream sales causes more homicides! However, in the real world, people often ask causal questions because answering these questions can influence decision-making. For example, you may ask:

• What did I eat that caused me stomach pain?
• Did raising the minimum wage cause a decrease in employment?
• What caused Americans to rebel against the British?

Amongst these questions of interest, which ones can we answer through math? How do we think about causation when we can only measure association? In this blog post, I will provide a gentle introduction on how to ask causal questions and walk through how to do causal inference.

By definition, causality addresses the direct effect of an intervention on the outcome. This idea stems from scientific experiments, where the scientist measures the change in the outcome in a controlled environment by intervening on a single component. The same experimental mindset applies when measuring outcomes in health and social sciences. While there are many approaches to set up causal inference,¹ I will address the potential outcomes framework that focuses on the experimentalist view of this topic.^2,3 Before jumping into formal notations and discussion how to make causal inference with an example, we first need to think about how to begin with a question causal inference can even solve. I will discuss causal inference from a simple example and one framework. However, there is much literature addressing more complicated methods. I provide more resources at the end of the post for those interested in reading further. My discussion presented here is largely based on Prof. Ding, whom I learned from, and his causal inference course.⁴

Specifying the Causal Question

The first step of asking a good causal question starts with identifying the intervention and outcome of interest. Some examples of intervention are policies, medical therapies, and educational programs. Medicine is the most common archetype of an intervention, often referred to as treatment, answering questions such as does taking this medicine relieve my pain? 

Translating a general question, as such the one posed above, into something that we can actually use causal methods on requires being as specific as possible for every aspect of the problem. Importantly, to interpret a causal effect heavily relies on domain-expert inputs. This not only includes the design element – what is the intervention and how is it randomized – but also what is the target population of interest. One consideration is the unit of interest in the analysis, for example, in educational studies, the units can be individual students, the classes, or the school as a whole. Careful selection criteria to include and exclude units of interest is essential to causal inference as well. Units should be excluded from the experiment if they cannot possibly receive the treatment. Taking the medicine example, if the person stopped feeling pain prior to taking the medicine, then they should be excluded from the study. Since we are considering the direct cause-and-effect relationship between the medicine and pain relief, the time ordering is very important! Moreover, if there are multiple types or doses of medicine – the choice is important to identify which one to consider. These decisions (e.g., taking 200mg or 400mg of ibuprofen) are important, especially when providing recommendations to other people in the target population not in the experiment. Although I will refer to this more generally as “medicine” and “pain relief” in the text following, these terms in practice should be measured with more care and consideration for an interpretable causal effect.

Formalizing the Problem

Writing “what if?” as a Science Table

We use this question – Does this medicine relieve pain? – as an example. Let us imagine an experiment where we recruit people experiencing pain as participants. Following the scientific method, we want to provide the two groups (those to take medicine and those to take placebo) the same environment. Note that how the groups are assigned needs to be random for causal interpretation (more on that later). Ideally, we want to give each person (unit, in this case) both medicine and the placebo to be able to tell whether taking the medicine did help this person relieve pain. However, we cannot observe both pain levels taking medicine and taking placebo. This is the fundamental problem of causal inference – both interventions cannot be assigned to the same unit. Causal inference is much like a thought experiment where if we were able to observe both outcomes (pain levels) at the same time.

To write this more formally, let us denote the experiment with the total number of participants of size n and the experimental units i, where i = 1,...,n. Let the intervention be binary (e.g., with medication, Z = 1, or with placebo, Z = 0). Then the potential outcomes can be written as Y_i(1) and Y_i(0). The full experiment can be written as a Science Table with both outcomes (pain levels) that are unobservable by nature. For example, we may ask the participants to rate their pain level from 1 to 10 after taking the medication or placebo. The Science Table may look like this:

One key assumption for writing the Science Table is the Stable Unit Treatment Value Assumption (SUTVA) consisting of (1) a no interference assumption, where each unit’s treatment is not dependent on another unit’s treatment and (2) a consistency assumption, where the treatment is well-defined. A well-defined treatment is one version of the treatment that is the same given to all units. The consistency follows the ideas proposed in the causal question section, where we want to be specific about what is the exact intervention. A well-defined outcome allows for clarity in separating those who received the intervention versus those who didn’t. The SUTVA also highlights another idea of units being independent from one another – this is important to consider since when this is violated, the direct effect of the intervention is difficult to calculate.

Defining the average treatment effect

Going back to our original research question on how medicine affects pain relief, the effect of interest is often the individual effect for each subject i – what is the difference if they took medicine minus if they took placebo? If we have the Science Table, this problem would be so simple! We can write this as Y_i(1) - Y_i(0) for each participant and be done! From the example above we can see that the medication reduces the pain by 1. We would then interpret this pain reduction of 1 caused by the medication directly. However, since this is not feasible, we may want to consider the average reduction in pain for each treatment, for all the participants who took them. The observed difference between the average treatment, under the above assumptions, can be interpreted as causal. The average effect of the treatment on the outcome (average treatment effect) can be written as expectation of the average Y_i(1) minus average of Y_i(0). Calculating this causal quantity is an active area of research – especially on how variable the estimate should be. I will provide an example of using regression to estimate the ATE.

Example with Linear Regression in Randomized Experiment

Randomized intervention for comparable groups

One important assumption and condition for causal interpretation we are operating under is the “randomized” experiment element. Specifically, this refers to an experiment where the intervention is being randomized completely. The randomization is the key to making the treated and control groups comparable! More formally, this step provides the same probability of treatment to different types of people, thus making the two groups similar on potential confounders. For example, if older people tend to take more medication and get more pain than younger people, then age – a confounder – could explain away the medication’s effect on pain. Randomizing would, in general, prevent a treatment group from being largely older people. Having comparable groups is essential to assigning causality for the interpretation.

Calculating with linear regression

The goal of this section is to provide an example of a common approach to calculating the average treatment effect and its variance. Taking the data, we can calculate the average treatment effect as a regression problem by fitting the treatment Z on the outcome Y. Recall in the example, the treatment is either medication, Z=1, or placebo, Z=0. Fitting the observed data using a linear regression will result in a form: Y = mZ + b, where b is the intercept and m is the slope. To interpret these quantities, we can plug in the value of Z: when treatment is the medication, Z = 1, the equation is written as Y = mZ + b; when the treatment is the placebo, Z = 0, the equation is written as Y = b. The difference is only m, which is the average treatment effect. The b represents the average pain outcome. Note that the variance calculation cannot be taken directly for m, but the robust version needs to be used for interpretation.⁴

Takeaway

Causal inference may seem unapproachable due to traditional understanding of statistics. More importantly, specifying a causal question of interest requires providing sufficient detail to write the problem down mathematically. It is extremely important to be cautious about interpretation when making causal inferences.

If you’d like to read more about causal inference, check out the following resources:

• A First Course in Causal Inference
• Introduction to Modern Causal Inference
• A Causal Roadmap for Generating High-Quality Real-World Evidence

References

1. Yao L, Chu Z, Li S, Li Y, Gao J, Zhang A. A survey on causal inference. ACM Transactions on Knowledge Discovery from Data (TKDD). 2021 May 10;15(5):1-46.
2. Splawa-Neyman J, TomX RN. Pr6ba uzasadnienia zastosowafi rachunku prawdopodobiefistwa do doswiadcze'n polowych. Roczniki Nauk Rolniczych. 1923;10:1-51.
3. Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology. 1974 Oct;66(5):688.
4. Ding P. A First Course in Causal Inference. arXiv preprint arXiv:2305.18793. 2023 May 30.