Learning Causal Abstractions of Linear Structural Causal Models

Causal Abstraction formalizes the property of distinct causal models to describe the same phenomenon with different levels of detail [Beckers and Halpern, 2019]. Despite having different variables and mechanisms, whenever two Structural Causal Models (SCMs) are in an abstraction relation, there must always exist at least one implementation on the low-level concrete model of any property of the high-level abstract one — such as values, interventions, mechanisms, and endogenous or exogenous distributions.

Abstract causal models allow the interpretation of causal models with large number of variables, such as in climate phenomena [Chalupka et al., 2016] or brain activation patterns [Dubois et al., 2020]. Causal Abstraction has also found wide interest in explainable AI to align machine representations with human-interpretable concepts in feedforward neural networks [Geiger et al., 2021, Geiger et al., 2023], concept-based neural networks [Marconato et al., 2023], and Large Language Models [Wu et al., 2024, Geiger et al., 2024].

Previous works on the definition of Causal Abstraction do not focus on the graphical and parametrical conditions for two models to be in an abstraction relation. Furthermore, the problem of learning abstractions from data, when the high-level model is not known, is still open. In this context, Zennaro [2022] and Geiger et al. [2023] propose methods to learn an abstraction function assuming to know both the low-level and the abstract model, while Chalupka et al. [2016], Kekić et al. [2023] and Felekis et al. [2024] assume to have at least the graphical structure of the abstract model.

In this paper, we tackle these issues by focusing on the scenario where two linear SCMs are abstracted by a linear transformation, as shown in Figure 1. In particular, we study necessary and sufficient conditions for abstraction in terms of the edges and the coefficients of the models. We then propose Abs-LiNGAM, a strategy to learn from data the abstract model, the concrete model, and their abstraction function under the further assumption of non-Gaussian exogenous noise. We summarize our contributions as follows:

1. 1.

   We first prove that abstract edges necessarily require edges in the low-level model to connect relevant variables, i.e., variables on which the abstraction function directly depends (Theorem 1). Then, we show that the abstraction necessarily arranges concrete variables in adjacent and disjoint blocks that must follow the abstract causal ordering (Theorem 2).

2. 2.

   We then prove a necessary and sufficient condition for causal abstraction that relates the coefficients of the linear models and the abstraction function (Theorem 3). In this way, we can characterize the set of all concrete models that are abstracted by a given abstract SCM and design a complete and correct algorithm to sample any model from this set (Algorithm 1).

3. 3.

   We introduce Abs-LiNGAM, a method to speedup the causal discovery of large linear non-Gaussian models given an additional and small dataset sampled from the observational joint distribution of the model and one of its abstractions. Abs-LiNGAM recovers the abstraction function, learns the abstract model using low-level data, and finally constrains the recovery of the concrete model by ensuring that the necessary conditions we introduced are satisfied (Algorithm 2).

4. 4.

   As we report in Section 5, experiments in simulated settings show that Abs-LiNGAM substantially reduces the search space, and thus the execution time, compared to directly solving the problem on the low-level dataset with DirectLiNGAM [Shimizu et al., 2011].

We also publicly release online the code of Abs-LiNGAM and the experimental settings¹¹ 1 https://github.com/rmassidda/causabs.

Given a set of variables $\bm{{X}}$, we denote the domain of each variable ${X\in\bm{{X}}}$ as $\mathcal{D}(X)$ and of any subset ${\bm{{V}}\subseteq\bm{{X}}}$ as ${\mathcal{D}(\bm{{V}})}$. We define a Structural Causal Model [Pearl, 2009, SCM;] as a tuple ${\mathcal{M}=\left(\bm{{X}},\bm{{E}},{\{f_{X}\}}_{X\in\bm{{X}}},\mathbb{{P}}_{\bm{{E}}}\right)}$, where

1. 1.

   $\bm{{X}}$ is a set containing $d$ distinct endogenous variables,

2. 2.

   $\bm{{E}}$ is a set containing $d$ distinct exogenous variables,

3. 3.

   ${f_{X}\colon\mathcal{D}(\operatorname{Pa}\left\lparen X\right\rparen\cup\{E_{X}\})\to\mathcal{D}(X)}$ is a causal mechanism, i.e. a function that determines the value of the variable $X\in\bm{{X}}$ given its parents $\operatorname{Pa}\left\lparen X\right\rparen$ and the exogenous noise term $E_{X}\in\bm{{E}}$,

4. 4.

   $\mathbb{{P}}_{\bm{{E}}}$ is the joint distribution over $\bm{{E}}$.

We assume that parental relations define a directed acyclic graph $\mathcal{G_{M}}$ and, consequently, that the reduced form of the model always has a unique solution [Bongers et al., 2021]. By slightly abusing the notation, we denote as $\mathcal{M}$ both the SCM and its reduced form $\mathcal{M}\colon\mathcal{D}(\bm{{E}})\to\mathcal{D}(\bm{{X}})$ mapping exogenous to endogenous values. A hard intervention is an assignment $i=(\bm{{V}}\leftarrow\bm{{v}})$ on a subset of variables $\bm{{V}}\subseteq\bm{{X}}$ that replaces each mechanism of the variables $\bm{{V}}$ with a constant value $\bm{{v}}\in\mathcal{D}(\bm{{V}})$. We denote as $\bm{{I}}^{\ast}$ the set of all hard interventions on an SCM, containg all possible assignments to any subset of endogenous variables, also including the empty intervention. Formally, an intervention $i$ results in a different SCM $\mathcal{M}^{i}$ defined by the tuple ${\left(\bm{{X}},\bm{{E}},{\{f_{X}^{i}\}}_{X\in\bm{{X}}},\mathbb{{P}}_{\bm{{E}}}\right)}$, where ${f_{X}^{i}=f_{X}}$ if $X\notin\bm{{V}}$ and $f_{X}^{i}(\cdot)=v_{X}$ otherwise. We then define the restriction of an intervened causal model as the set of values that the model can take after the intervention, which we denote as

where $\operatorname{Rst}\left(\mathcal{M}\right)=\mathcal{D}(\bm{{X}})$ for a non-intervened SCM.

We assume faithfulness and causal sufficiency, i.e., the absence of hidden confounding and selection bias [Spirtes et al., 2000]. In particular, faithfulness implies the absence of canceling paths across variables, while causal sufficiency implies mutual independence of exogenous terms, as in $E_{1}\perp\!\!\!\perp E_{2}$ for any $E_{1},E_{2}\in\bm{{E}}$.

A linear SCM ${\mathcal{M}=\left(\bm{{X}},\bm{{E}},{\bm{\mathbf{W}}},\mathbb{{P}}_{\bm{{E}}}\right)}$, also known as a linear Additive Noise Model (ANM) [Peters et al., 2017], is an SCM whose structural equations are linear and represented by an upper-triangular adjacency matrix ${{\bm{\mathbf{W}}}\in\real^{d\times d}}$, as in

We can compute the reduced form of the model in closed form as,

where ${\bm{\mathbf{F}}}={({\bm{\mathbf{I}}}-{\bm{\mathbf{W}}})}^{-1}$ is a $d\times d$ linear transformation.

Causal Abstraction theory relates variables across different SCMs to determine whether they represent in a consistent way the same system at different levels of detail [Beckers and Halpern, 2019]. Overall, we refer to concrete, or low-level, causal models as $\mathcal{L}=(\bm{{X}},\bm{{E}},\bm{{f}},\mathbb{{P}}_{\bm{{E}}})$ and to abstract, or high-level, causal models as $\mathcal{H}=(\bm{{Y}},\bm{{U}},\bm{{g}},\mathbb{{P}}_{\bm{{U}}})$, where ${|\bm{{X}}|\geq|\bm{{Y}}|}$. An abstraction requires to consider two subsets of allowed interventions $\bm{{I}}\subseteq\bm{{I}}^{\ast}$ and $\bm{{J}}\subseteq\bm{{J}}^{\ast}$ respectively on the concrete and the abstract model.

In this work, we focus on strong abstractions, where any concrete or abstract intervention is allowed, i.e., $\bm{{I}}=\bm{{I}}^{\ast}$ and $\bm{{J}}=\bm{{J}}^{\ast}$. Then, given a surjective function $\tau\colon\mathcal{D}(\bm{{X}})\to\mathcal{D}(\bm{{Y}})$, $\mathcal{H}$ is a $\tau$-abstraction of $\mathcal{L}$ if and only if there exists a surjective function $\gamma\colon\mathcal{D}(\bm{{E}})\to\mathcal{D}(\bm{{U}})$ on the exogenous variables such that, for any low-level intervention $i\in\bm{{I}}$ and any exogenous configuration $\bm{{e}}\in\mathcal{D}(\bm{{E}})$, it holds

where the intervention map $\omega\colon\bm{{I}}\to\bm{{J}}$ is uniquely induced by the value abstraction function $\tau$ [Massidda et al., 2023]. Formally, $\omega(i)=j$ if and only if $\operatorname{Rst}\left(\mathcal{H}^{j}\right)=\tau(\operatorname{Rst}\left(\mathcal{L}^{i}\right))$ and $j\in\bm{{J}}$, otherwise $\omega(i)$ is undefined. We refer to Equation 4 as the interventional consistency property and to the function $\gamma$ as the exogenous abstraction function. Similarly, we refer to the equation

on non-intervened models as observational consistency. Since the empty intervention is necessarily a fixed point of the intervention map [Massidda et al., 2023], interventional consistency implies observational consistency.

Finally, as for the causal mechanisms, we assume that the abstraction function does not yield cancelling paths towards abstract variables. Formally, the composition of the abstraction function $\tau$, with the concrete model $\mathcal{L}$ must not cancel the effect of concrete variables $\bm{{X}}$ on abstract variables $\bm{{Y}}$.

In this section, we study graphical and structural properties of linear causal models in a linear abstraction relation. First, we prove that the set of concrete relevant variables on which each abstract variable depends are necessarily disjoint (Section 3.1). Further, we prove necessary and sufficient conditions on the existence of an abstract edge in terms of the directed paths between relevant variables in the concrete graph (Section 3.2). We then show that the abstraction function constrains the causal ordering of concrete blocks composed of both relevant and non-relevant variables (Section 3.3). Finally, given the notion of concrete block, we provide an equivalent formulation of abstraction based on the model parameters, which also characterizes the set of possible concretizations of an abstract model (Section 3.4).

In this section, we introduce Abs-LiNGAM (Algorithm 2), a strategy to exploit our results on ${\bm{\mathbf{T}}}$-abstraction to speedup observational causal discovery of linear non-Gaussian models, e.g. LiNGAM [Shimizu et al., 2011]. The intuition is that whenever we have a ${\bm{\mathbf{T}}}$-abstraction of an unknown model to learn, we can exclude all the candidate solutions not satisfying the graphical conditions we presented in the previous sections. Furthermore, in Abs-LiNGAM, we demonstrate how to infer prior knowledge for the concrete model from a small number of paired concrete-abstract samples, even when the abstract model and the abstraction function are unknown, and an abstract dataset is not directly available. In the following, we formalize the data-generation process and the steps of Abs-LiNGAM.

In this section, we discuss our analysis on the performance of Abs-LiNGAM (Algorithm 2) on simulated data. In particular, we validate whether a small amount of paired concrete-abstract observations can reduce the search space, and thus the execution time, of DirectLiNGAM [Shimizu et al., 2011], without compromising the quality of the retrieved concrete causal structure. As baseline, we compare against applying DirectLiNGAM to the concrete dataset without any abstract-induced prior knowledge.

For each run, we sample the parameters of an abstraction function and of an abstract linear SCM. We then generate a concrete causal model by sampling one of the possible ${\bm{\mathbf{T}}}$-concretizations of the abstract model with Algorithm 1. We provide details on our experimental setup and additional results respectively in Appendix C and in Appendix E.

We study the performance of Abs-LiNGAM for an increasing number of paired samples (Figure 2(a)). Since Abs-LiNGAM is a multi-step algorithm, the quality of the retrieved concrete causal model strictly depends on the correctness of the abstraction function, the consequent generated abstract data and abstract causal discovery. As expected, whenever the size of the paired dataset $|\mathcal{D}_{J}|$ is too small, Abs-LiNGAM wrongly identifies concrete paths as forbidden and, compared to the baseline, fails to retrieve the correct concrete causal model. However, whenever the number of paired samples approaches the number of concrete nodes $|\bm{{X}}|$, Abs-LiNGAM performs similarly to the baseline and correctly retrieves the concrete causal model. We observe the same trend for concrete graphs of increasing size (Figure 2(b)), highlighting how prior knowledge induced from the abstract model significantly reduces the execution time compared to the baseline.

Furthermore, we found that bootstrapping abstract causal discovery, i.e., aggregating several iterations on randomly extracted sub-datasets, improves the performance on the downstream concrete discovery task without noticeably affecting the execution time, which is still dominated by the final concrete causal discovery run.

Several works addressed the problem of clustering together variables to reduce dimensionality and maintain the identifiability of causal effect. Both Anand et al. [2023] and Wahl et al. [2023] deal with the problem of partitioning a causal graph into clusters where causal relations at the micro-level are translated as causal edges at the macro-level. Tikka et al. [2023] study instead a particular class of groups, which they define as transit clusters, where only part of the variables are allowed to have ingoing or outgoing edgs from the cluster.

Differently from previous works, our work focuses instead on the necessary conditions for causal abstraction and results in different definitions for the grouping of micro-variables. It is however an interesting direction to assess whether different assumptions, for instance on the intervention map, might lead to comparable definitions.

In parallel, several recent papers explored the problem of fitting an abstraction function from data by focusing on either discrete [Zennaro et al., 2023, Felekis et al., 2024] or linear [Kekić et al., 2023, Geiger et al., 2024] SCMs. Notably, apart from interventional samples, all these works assume to have at least partial knowledge of the graphs, the intervention map, or the set of concrete relevant variables corresponding to each abstract one.

Based on our theoretical results on the graphical and parametric conditions of linear abstraction for linear causal models, we instead propose to learn both the abstract and the concrete model, and their abstraction function directly from observational data and without any prior knowledge or any constraint on the graphical structure of the two models.

In this paper, we studied the necessary and sufficient conditions on the causal ordering and the parameters for two linear Structural Causal Models to be in a linear abstraction. Furthermore, we introduced the first procedure to sample from the set of all possible concretizations of an abstract SCM, which can be used in other abstraction applications.

We also proposed Abs-LiNGAM, a strategy to speedup causal discovery of a linear non-Gaussian concrete causal model given an additional dataset of paired observations on concrete and abstract variables. Finally, we empirically highlighted how Abs-LiNGAM leverages abstract information to reduce the search space and improve execution time without sacrificing on the quality of the discovered structure.

An interesting direction for future work is to extend our results to non-linear models and non-linear abstraction functions, and to tackle the causal sufficiency assumption, which requires full-observability of the concrete realizations.