Social scientists and policy makers continue to put increased emphasis on identifying causal effects in their research, employing the myriad of novel approaches that have been developed in recent years. With this rise in the use of causal analysis tools, the importance of understanding the assumptions underlying such methods has increased. Methods and estimates are occasionally stretched beyond the limits of the underlying assumptions. This has increased the need for a thorough understanding of what assumptions are necessary to identify causal estimates outside of a purely experimental framework. Additionally, it has lead to a need for validation tests that allow researchers to provide sound evidence that estimates are unbiased. This dissertation focuses on one area, methods for increasing the external validity of experimental estimates, of interest to researchers and policy makers alike. This problem is addressed from multiple angles, both theoretical, practical, and from a design perspective. First, I outline the assumptions necessary to identify population treatment effects from experimental data. I then discuss a new estimating strategy for testing the key identifying assumption central to most causal methods, the notion of exchangeability. This new method leverages tests of equivalence to redefine the approach to validation tests such as balance and placebo tests. Finally, a new approach to improving the underlying data from which we explore questions and estimate quantities of interest is provided. This new method, response rate sampling, allows researchers to collect more representative survey data. Combined, these tools will allow researchers to move more fluidly between experimental and observational estimates.

Explains the value of quasi-experimental techniques that can be used to approximate randomized experiments. The goal is to describe the logic of causal inference for researchers and policymakers who are not necessarily trained in experimental and quasi-experimental designs and statistical techniques.

This User’s Guide is a resource for investigators and stakeholders who develop and review observational comparative effectiveness research protocols. It explains how to (1) identify key considerations and best practices for research design; (2) build a protocol based on these standards and best practices; and (3) judge the adequacy and completeness of a protocol. Eleven chapters cover all aspects of research design, including: developing study objectives, defining and refining study questions, addressing the heterogeneity of treatment effect, characterizing exposure, selecting a comparator, defining and measuring outcomes, and identifying optimal data sources. Checklists of guidance and key considerations for protocols are provided at the end of each chapter. The User’s Guide was created by researchers affiliated with AHRQ’s Effective Health Care Program, particularly those who participated in AHRQ’s DEcIDE (Developing Evidence to Inform Decisions About Effectiveness) program. Chapters were subject to multiple internal and external independent reviews. More more information, please consult the Agency website: www.effectivehealthcare.ahrq.gov)

Sections include: experiments and generalised causal inference; statistical conclusion validity and internal validity; construct validity and external validity; quasi-experimental designs that either lack a control group or lack pretest observations on the outcome; quasi-experimental designs that use both control groups and pretests; quasi-experiments: interrupted time-series designs; regresssion discontinuity designs; randomised experiments: rationale, designs, and conditions conducive to doing them; practical problems 1: ethics, participation recruitment and random assignment; practical problems 2: treatment implementation and attrition; generalised causal inference: a grounded theory; generalised causal inference: methods for single studies; generalised causal inference: methods for multiple studies; a critical assessment of our assumptions.

The statistics profession is at a unique point in history. The need for valid statistical tools is greater than ever; data sets are massive, often measuring hundreds of thousands of measurements for a single subject. The field is ready to move towards clear objective benchmarks under which tools can be evaluated. Targeted learning allows (1) the full generalization and utilization of cross-validation as an estimator selection tool so that the subjective choices made by humans are now made by the machine, and (2) targeting the fitting of the probability distribution of the data toward the target parameter representing the scientific question of interest. This book is aimed at both statisticians and applied researchers interested in causal inference and general effect estimation for observational and experimental data. Part I is an accessible introduction to super learning and the targeted maximum likelihood estimator, including related concepts necessary to understand and apply these methods. Parts II-IX handle complex data structures and topics applied researchers will immediately recognize from their own research, including time-to-event outcomes, direct and indirect effects, positivity violations, case-control studies, censored data, longitudinal data, and genomic studies.

This is a companion volume to Causal Models in the Social Sciences, the majority of articles concern panel designs involving repeated measurements while a smaller cluster involve discussions of how experimental designs may be improved by more explicit attention to causal models. All of the papers are concerned with complications that may occur in actual research designs- as compared with idealized ones that often become the basis of textbook discussions of design issues.

The third chapter, by adopting a data-driven theory discovery technique, proposes the hypothesis that the local government in China is more likely to respond if the petitioner sends a credible signal to the government that she is an insider. It further tests this hypothesis with an active-labeling-enhanced semi-supervised learning algorithm as proposed in this dissertation.

The modern proliferation of large observational databases -- in fields such as e-commerce and electronic health -- presents challenges and opportunities for applied researchers. Such data can contain rich information about causal effects of interest, but the effects can only be estimated if we make untestable assumptions and carefully model the assignment mechanism. Experimental data provides a "virtuous" counterpart for the purposes of inferring causal effects, but randomized trials are often limited in size and, consequentially, lack precision. In this thesis, we consider problems of "data fusion, " in which observational and experimental datasets are used together to estimate causal effects. The problem is considered from three angles. First, we develop methods for merging experimental and observational causal effect estimates in the case when all confounding variables are measured in the observational studies. Next, we remove the unconfoundedness assumption, which leads to a new class of estimators based on a shrinkage approach. Finally, we propose a novel solution for designing experiments informed by observational studies, making use of the regret minimization framework. Throughout, we deploy tools from disparate areas of the literature, including Empirical Bayes, decision theory, and convex optimization.