Home / Research / Talks / Teaching / Miscellanea

Research interests

• Causal inference
• Inference in the presence of interference
• Multiple testing and selective inference
• Statistical reinforcement learning

Publications

• Covariate Shift Corrected Conditional Randomization Test
  Bowen Xu, Yiwen Huang, Chuan Hong, Shuangning Li, Molei Liu.
  Conference on Neural Information Processing Systems (NeurIPS), 2024.
Conditional independence tests are crucial across various disciplines in determining the independence of an outcome variable Y from a treatment variable X, conditioning on a set of confounders Z. The Conditional Randomization Test (CRT) offers a powerful framework for such testing by assuming known distributions of X | Z; it controls the Type-I error exactly, allowing for the use of flexible, black-box test statistics. In practice, testing for conditional independence often involves using data from a source population to draw conclusions about a target population. This can be challenging due to covariate shift -- differences in the distribution of X, Z, and surrogate variables, which can affect the conditional distribution of Y∣X,Z -- rendering traditional CRT approaches invalid. To address this issue, we propose a novel Covariate Shift Corrected Pearson Chi-squared Conditional Randomization (csPCR) test. This test adapts to covariate shifts by integrating importance weights and employing the control variates method to reduce variance in the test statistics and thus enhance power. Theoretically, we establish that the csPCR test controls the Type-I error asymptotically. Empirically, through simulation studies, we demonstrate that our method not only maintains control over Type-I errors but also exhibits superior power, confirming its efficacy and practical utility in real-world scenarios where covariate shifts are prevalent. Finally, we apply our methodology to a real-world dataset to assess the impact of a COVID-19 treatment on the 90-day mortality rate among patients.
abstract arXiv



• Maxway CRT: Improving the Robustness of Model-X Inference
  Shuangning Li, Molei Liu.
  Journal of the Royal Statistical Society: Series B (JRSSB), 85(5), 2023.
The model-X conditional randomization test (CRT) proposed by Candès et al. (2018) is known as a flexible and powerful testing procedure for the conditional independence hypothesis: X is independent of Y conditional on Z. Though having many attractive properties, the model-X CRT relies on the model-X assumption that we have access to perfect knowledge of the distribution of X conditional on Z. If there is a specification error in modeling the distribution of X conditional on Z, this approach may lose its validity. This problem is even more severe when the adjustment covariates Z are of high dimensionality, in which situation precise modeling of X against Z can be hard. In response to this, we propose the Maxway (Model and Adjust X With the Assistance of Y) CRT, a more robust inference approach for conditional independence when the conditional distribution of X is unknown and needs to be estimated from the data. Besides the distribution of X | Z, the Maxway CRT also learns the distribution of Y | Z, using it to calibrate the resampling distribution of X to gain robustness to the error in modeling X. We show that the type-I error inflation of the Maxway CRT can be controlled by the learning error for the low-dimensional adjusting model plus the product of learning errors for the distribution of X | Z and the distribution of Y | Z. This result can be interpreted as an "almost doubly robust" property of the Maxway CRT. Through extensive simulation studies, we demonstrate that the Maxway CRT achieves significantly better type-I error control than existing model-X inference approaches while having similar power. Finally, we apply our methodology to the UK biobank dataset with the goal of studying the relationship between the functional SNP of statins and the risk for type II diabetes mellitus.
abstract paper arXiv code slides/poster



• Detecting Interference in A/B Testing with Increasing Allocation
  Kevin Han, Shuangning Li, Jialiang Mao, Han Wu.
  ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2023.
In the past decade, the technology industry has adopted online randomized controlled experiments (a.k.a. A/B testing) to guide product development and make business decisions. In practice, A/B tests are often implemented with increasing treatment allocation: the new treatment is gradually released to an increasing number of units through a sequence of randomized experiments. In scenarios such as experimenting in a social network setting or in a bipartite online marketplace, interference among units may exist, which can harm the validity of simple inference procedures. In this work, we introduce a widely applicable procedure to test for interference in A/B testing with increasing allocation. Our procedure can be implemented on top of an existing A/B testing platform with a separate flow and does not require a priori a specific interference mechanism. In particular, we introduce two permutation tests that are valid under different assumptions. Firstly, we introduce a general statistical test for interference requiring no additional assumption. Secondly, we introduce a testing procedure that is valid under a time fixed effect assumption. The testing procedure is of very low computational complexity, it is powerful, and it formalizes a heuristic algorithm implemented already in industry. We demonstrate the performance of the proposed testing procedure through simulations on synthetic data. Finally, we discuss one application at LinkedIn, where a screening step is implemented to detect potential interference in all their marketplace experiments with the proposed methods in the paper.
abstract paper arXiv slides/poster



• Transfer Learning in Genome-Wide Association Studies with Knockoffs
  Shuangning Li, Zhimei Ren, Chiara Sabatti, Matteo Sesia.
  Sankhya B, 2022.
This paper presents and compares alternative transfer learning methods that can increase the power of conditional testing via knockoffs by leveraging prior information in external data sets collected from different populations or measuring related outcomes. The relevance of this methodology is explored in particular within the context of genome-wide association studies, where it can be helpful to address the pressing need for principled ways to suitably account for, and efficiently learn from the genetic variation associated to diverse ancestries. Finally, we apply these methods to analyze several phenotypes in the UK Biobank data set, demonstrating that transfer learning helps knockoffs discover more numerous associations in the data collected from minority populations, potentially opening the way to the development of more accurate polygenic risk scores.
abstract paper arXiv code



• Random Graph Asymptotics for Treatment Effect Estimation under Network Interference
  Shuangning Li, Stefan Wager.
  Annals of Statistics (AOS), 50(4), 2022.
The network interference model for causal inference places all experimental units at the vertices of an undirected exposure graph, such that treatment assigned to one unit may affect the outcome of another unit if and only if these two units are connected by an edge. This model has recently gained popularity as means of incorporating interference effects into the Neyman--Rubin potential outcomes framework; and several authors have considered estimation of various causal targets, including the direct and indirect effects of treatment. In this paper, we consider large-sample asymptotics for treatment effect estimation under network interference in a setting where the exposure graph is a random draw from a graphon. When targeting the direct effect, we show that---in our setting---popular estimators are considerably more accurate than existing results suggest, and provide a central limit theorem in terms of moments of the graphon. Meanwhile, when targeting the indirect effect, we leverage our generative assumptions to propose a consistent estimator in a setting where no other consistent estimators are currently available. We also show how our results can be used to conduct a practical assessment of the sensitivity of randomized study inference to potential interference effects. Overall, our results highlight the promise of random graph asymptotics in understanding the practicality and limits of causal inference under network interference.
abstract paper arXiv code slides/poster video



• Cooperative Learning for Multiview Analysis
  Daisy Yi Ding, Shuangning Li, Balasubramanian Narasimhan, Robert Tibshirani.
  Proceedings of the National Academy of Sciences (PNAS), 119(38), 2022.
We propose a new method for supervised learning with multiple sets of features ("views"). The multi-view problem is especially important in biology and medicine, where "-omics" data such as genomics, proteomics and radiomics are measured on a common set of samples. Cooperative learning combines the usual squared error loss of predictions with an "agreement" penalty to encourage the predictions from different data views to agree. By varying the weight of the agreement penalty, we get a continuum of solutions that include the well-known early and late fusion approaches. Cooperative learning chooses the degree of agreement (or fusion) in an adaptive manner, using a validation set or cross-validation to estimate test set prediction error. One version of our fitting procedure is modular, where one can choose different fitting mechanisms (e.g. lasso, random forests, boosting, neural networks) appropriate for different data views. In the setting of cooperative regularized linear regression, the method combines the lasso penalty with the agreement penalty, yielding feature sparsity. The method can be especially powerful when the different data views share some underlying relationship in their signals that can be exploited to boost the signals. We show that cooperative learning achieves higher predictive accuracy on simulated data and real multiomics examples of labor onset prediction and breast ductal carcinoma in situ and invasive breast cancer classification. Leveraging aligned signals and allowing flexible fitting mechanisms for different modalities, cooperative learning offers a powerful approach to multiomics data fusion.
abstract paper arXiv code video



• Average Direct and Indirect Causal Effects under Interference
  Yuchen Hu, Shuangning Li, Stefan Wager.
  Biometrika, 109(4), 2022.
We propose a definition for the average indirect effect of a binary treatment in the potential outcomes model for causal inference under cross-unit interference. Our definition is analogous to the standard definition of the average direct effect, and can be expressed without needing to compare outcomes across multiple randomized experiments. We show that the proposed indirect effect satisfies a decomposition theorem whereby, in a Bernoulli trial, the sum of the average direct and indirect effects always corresponds to the effect of a policy intervention that infinitesimally increases treatment probabilities. We also consider a number of parametric models for interference, and find that our (non-parametric) indirect effect remains a natural estimand when re-expressed in the context of these models.
abstract paper arXiv



• Searching for Robust Associations with a Multi-Environment Knockoff Filter
  Shuangning Li, Matteo Sesia, Yaniv Romano, Emmanuel Candès, Chiara Sabatti.
  Biometrika, 109(3), 2022.
This paper develops a method based on model-X knockoffs to find conditional associations that are consistent across diverse environments, controlling the false discovery rate. The motivation for this problem is that large data sets may contain numerous associations that are statistically significant and yet misleading, as they are induced by confounders or sampling imperfections. However, associations consistently replicated under different conditions may be more interesting. In fact, consistency sometimes provably leads to valid causal inferences even if conditional associations do not. While the proposed method is flexible and can be deployed in a wide range of applications, this paper highlights its relevance to genome-wide association studies, in which consistency across populations with diverse ancestries mitigates confounding due to unmeasured variants. The effectiveness of this approach is demonstrated by simulations and applications to the UK Biobank data.
abstract paper arXiv code slides/poster video



• Sharp Bounds for Exponential Approximations of NWUE Distributions
  Mark Brown, Shuangning Li.
  Methodology and Computing in Applied Probability, 20(3), 2017.
Let F be an NWUE distribution with mean 1 and G be the stationary renewal distribution of F. We would expect G to converge in distribution to the unit exponential distribution as its mean goes to 1. In this paper, we derive sharp bounds for the Kolmogorov distance between G and the unit exponential distribution, as well as between G and an exponential distribution with the same mean as G. We apply the bounds to geometric convolutions and to first passage times.
abstract paper slides/poster

Preprints

• Dyadic Reinforcement Learning
  Shuangning Li, Lluis Salvat Niell, Sung Won Choi, Inbal Nahum-Shani, Guy Shani, Susan Murphy.
  arXiv:2308.07843.   2023.
Mobile health aims to enhance health outcomes by delivering interventions to individuals as they go about their daily life. The involvement of care partners and social support networks often proves crucial in helping individuals managing burdensome medical conditions. This presents opportunities in mobile health to design interventions that target the dyadic relationship---the relationship between a target person and their care partner---with the aim of enhancing social support. In this paper, we develop dyadic RL, an online reinforcement learning algorithm designed to personalize intervention delivery based on contextual factors and past responses of a target person and their care partner. Here, multiple sets of interventions impact the dyad across multiple time intervals. The developed dyadic RL is Bayesian and hierarchical. We formally introduce the problem setup, develop dyadic RL and establish a regret bound. We demonstrate dyadic RL's empirical performance through simulation studies on both toy scenarios and on a realistic test bed constructed from data collected in a mobile health study.
abstract arXiv code data slides/poster



• Experimenting under Stochastic Congestion
  Shuangning Li, Ramesh Johari, Stefan Wager, Kuang Xu.
  arXiv:2302.12093.   2023.
We study randomized experiments in a service system when stochastic congestion can arise from temporarily limited supply and/or demand. Such congestion gives rise to cross-unit interference between the waiting customers, and analytic strategies that do not account for this interference may be biased. In current practice, one of the most widely used ways to address stochastic congestion is to use switchback experiments that alternatively turn a target intervention on and off for the whole system. We find, however, that under a queueing model for stochastic congestion, the standard way of analyzing switchbacks is inefficient, and that estimators that leverage the queueing model can be materially more accurate. We also consider a new class of experimental design, which can be used to estimate a policy gradient of the dynamic system using only unit-level randomization, thus alleviating key practical challenges that arise in running a switchback.
abstract arXiv slides/poster



• Network Interference in Micro-Randomized Trials
  Shuangning Li, Stefan Wager.
  arXiv:2202.05356.   2022.
The micro-randomized trial (MRT) is an experimental design that can be used to develop optimal mobile health interventions. In MRTs, interventions in the form of notifications or messages are sent through smart phones to individuals, targeting a health-related outcome such as physical activity or weight management. Often, mobile health interventions have a social media component; an individual's outcome could thus depend on other individuals' treatments and outcomes. In this paper, we study the micro-randomized trial in the presence of such cross-unit interference. We model the cross-unit interference with a network interference model; the outcome of one individual may affect the outcome of another individual if and only if they are connected by an edge in the network. Assuming the dynamics can be represented as a Markov decision process, we analyze the behavior of the outcomes in large sample asymptotics and show that they converge to a mean-field limit when the sample size goes to infinity. Based on the mean-field result, we give characterization results and estimation strategies for various causal estimands including the short-term direct effect of a binary intervention, its long-term direct effect and its long-term total effect.
abstract arXiv slides/poster



• Deploying the Conditional Randomization Test in High Multiplicity Problems
  Shuangning Li, Emmanuel Candès.
  arXiv:2110.02422.   2021.
This paper introduces the sequential CRT, which is a variable selection procedure that combines the conditional randomization test (CRT) and Selective SeqStep+. Valid p-values are constructed via the flexible CRT, which are then ordered and passed through the selective SeqStep+ filter to produce a list of discoveries. We develop theory guaranteeing control on the false discovery rate (FDR) even though the p-values are not independent. We show in simulations that our novel procedure indeed controls the FDR and are competitive with -- and sometimes outperform -- state-of-the-art alternatives in terms of power. Finally, we apply our methodology to a breast cancer dataset with the goal of identifying biomarkers associated with cancer stage.
abstract arXiv code slides/poster video