The availability of microdata has increased rapidly over the last decades, and standard statistical and econometric software packages for data analysis include ever more sophisticated modeling options. The goal of this book is to familiarize readers with a wide range of commonly used models, and thereby to enable them to become critical consumers of current empirical research, and to conduct their own empirical analyses. The focus of the book is on regression-type models in the context of large cross-section samples. In microdata applications, dependent variables often are qualitative and discrete, while in other cases, the sample is not randomly drawn from the population of interest and the dependent variable is censored or truncated. Hence, models and methods are required that go beyond the standard linear regression model and ordinary least squares. Maximum li- lihood estimation of conditional probability models and marginal probability e?ects are introduced here as the unifying principle for modeling, estimating and interpreting microdata relationships. We consider the limitation to m- imum likelihood sensible, from a pedagogical point of view if the book is to be used in a semester-long advanced undergraduate or graduate course, and from a practical point of view because maximum likelihood estimation is used in the overwhelming majority of current microdata research. In order to introduce and explain the models and methods, we refer to a number of illustrative applications. The main examples include the deter- nants of individual fertility, the intergenerational transmission of secondary schoolchoices,andthewageelasticityoffemalelaborsupply.

This book on statistical disclosure control presents the theory, applications and software implementation of the traditional approach to (micro)data anonymization, including data perturbation methods, disclosure risk, data utility, information loss and methods for simulating synthetic data. Introducing readers to the R packages sdcMicro and simPop, the book also features numerous examples and exercises with solutions, as well as case studies with real-world data, accompanied by the underlying R code to allow readers to reproduce all results. The demand for and volume of data from surveys, registers or other sources containing sensible information on persons or enterprises have increased significantly over the last several years. At the same time, privacy protection principles and regulations have imposed restrictions on the access and use of individual data. Proper and secure microdata dissemination calls for the application of statistical disclosure control methods to the da ta before release. This book is intended for practitioners at statistical agencies and other national and international organizations that deal with confidential data. It will also be interesting for researchers working in statistical disclosure control and the health sciences.

In the light of better and more detailed administrative databases, this open access book provides statistical tools for evaluating the effects of public policies advocated by governments and public institutions. Experts from academia, national statistics offices and various research centers present modern econometric methods for an efficient data-driven policy evaluation and monitoring, assess the causal effects of policy measures and report on best practices of successful data management and usage. Topics include data confidentiality, data linkage, and national practices in policy areas such as public health, education and employment. It offers scholars as well as practitioners from public administrations, consultancy firms and nongovernmental organizations insights into counterfactual impact evaluation methods and the potential of data-based policy and program evaluation.

Development Research in Practice leads the reader through a complete empirical research project, providing links to continuously updated resources on the DIME Wiki as well as illustrative examples from the Demand for Safe Spaces study. The handbook is intended to train users of development data how to handle data effectively, efficiently, and ethically. “In the DIME Analytics Data Handbook, the DIME team has produced an extraordinary public good: a detailed, comprehensive, yet easy-to-read manual for how to manage a data-oriented research project from beginning to end. It offers everything from big-picture guidance on the determinants of high-quality empirical research, to specific practical guidance on how to implement specific workflows—and includes computer code! I think it will prove durably useful to a broad range of researchers in international development and beyond, and I learned new practices that I plan on adopting in my own research group.†? —Marshall Burke, Associate Professor, Department of Earth System Science, and Deputy Director, Center on Food Security and the Environment, Stanford University “Data are the essential ingredient in any research or evaluation project, yet there has been too little attention to standardized practices to ensure high-quality data collection, handling, documentation, and exchange. Development Research in Practice: The DIME Analytics Data Handbook seeks to fill that gap with practical guidance and tools, grounded in ethics and efficiency, for data management at every stage in a research project. This excellent resource sets a new standard for the field and is an essential reference for all empirical researchers.†? —Ruth E. Levine, PhD, CEO, IDinsight “Development Research in Practice: The DIME Analytics Data Handbook is an important resource and a must-read for all development economists, empirical social scientists, and public policy analysts. Based on decades of pioneering work at the World Bank on data collection, measurement, and analysis, the handbook provides valuable tools to allow research teams to more efficiently and transparently manage their work flows—yielding more credible analytical conclusions as a result.†? —Edward Miguel, Oxfam Professor in Environmental and Resource Economics and Faculty Director of the Center for Effective Global Action, University of California, Berkeley “The DIME Analytics Data Handbook is a must-read for any data-driven researcher looking to create credible research outcomes and policy advice. By meticulously describing detailed steps, from project planning via ethical and responsible code and data practices to the publication of research papers and associated replication packages, the DIME handbook makes the complexities of transparent and credible research easier.†? —Lars Vilhuber, Data Editor, American Economic Association, and Executive Director, Labor Dynamics Institute, Cornell University

Introduction.Big data for twenty-first-century economic statistics: the future is now /Katharine G. Abraham, Ron S. Jarmin, Brian C. Moyer, and Matthew D. Shapiro --Toward comprehensive use of big data in economic statistics.Reengineering key national economic indicators /Gabriel Ehrlich, John Haltiwanger, Ron S. Jarmin, David Johnson, and Matthew D. Shapiro ;Big data in the US consumer price index: experiences and plans /Crystal G. Konny, Brendan K. Williams, and David M. Friedman ;Improving retail trade data products using alternative data sources /Rebecca J. Hutchinson ;From transaction data to economic statistics: constructing real-time, high-frequency, geographic measures of consumer spending /Aditya Aladangady, Shifrah Aron-Dine, Wendy Dunn, Laura Feiveson, Paul Lengermann, and Claudia Sahm ;Improving the accuracy of economic measurement with multiple data sources: the case of payroll employment data /Tomaz Cajner, Leland D. Crane, Ryan A. Decker, Adrian Hamins-Puertolas, and Christopher Kurz --Uses of big data for classification.Transforming naturally occurring text data into economic statistics: the case of online job vacancy postings /Arthur Turrell, Bradley Speigner, Jyldyz Djumalieva, David Copple, and James Thurgood ;Automating response evaluation for franchising questions on the 2017 economic census /Joseph Staudt, Yifang Wei, Lisa Singh, Shawn Klimek, J. Bradford Jensen, and Andrew Baer ;Using public data to generate industrial classification codes /John Cuffe, Sudip Bhattacharjee, Ugochukwu Etudo, Justin C. Smith, Nevada Basdeo, Nathaniel Burbank, and Shawn R. Roberts --Uses of big data for sectoral measurement.Nowcasting the local economy: using Yelp data to measure economic activity /Edward L. Glaeser, Hyunjin Kim, and Michael Luca ;Unit values for import and export price indexes: a proof of concept /Don A. Fast and Susan E. Fleck ;Quantifying productivity growth in the delivery of important episodes of care within the Medicare program using insurance claims and administrative data /John A. Romley, Abe Dunn, Dana Goldman, and Neeraj Sood ;Valuing housing services in the era of big data: a user cost approach leveraging Zillow microdata /Marina Gindelsky, Jeremy G. Moulton, and Scott A. Wentland --Methodological challenges and advances.Off to the races: a comparison of machine learning and alternative data for predicting economic indicators /Jeffrey C. Chen, Abe Dunn, Kyle Hood, Alexander Driessen, and Andrea Batch ;A machine learning analysis of seasonal and cyclical sales in weekly scanner data /Rishab Guha and Serena Ng ;Estimating the benefits of new products /W. Erwin Diewert and Robert C. Feenstra.

In a time when increasingly larger and complex data collections are being produced, it is clear that new and adaptive forms of data representation and analysis have to be conceived and implemented. Distributional data, i.e., data where a distribution rather than a single value is recorded for each descriptor, on each unit, come into this framework. Distributional data may result from the aggregation of large amounts of open/collected/generated data, or it may be directly available in a structured or unstructured form, describing the variability of some features. This book provides models and methods for the representation, analysis, interpretation, and organization of distributional data, taking into account its specific nature, and not relying on a reduction to single values, to be conform to classical paradigms. Conceived as an edited book, gathering contributions from multiple authors, the book presents alternative representations and analysis’ methods for distributional data of different types, and in particular, -Uni- and bi-variate descriptive statistics for distributional data -Clustering and classification methodologies -Methods for the representation in low-dimensional spaces -Regression models and forecasting approaches for distribution-valued variables Furthermore, the different chapters -Feature applications to show how the proposed methods work in practice, and how results are to be interpreted, -Often provide information about available software. The methodologies presented in this book constitute cutting-edge developments for stakeholders from all domains who produce and analyse large amounts of complex data, to be analysed in the form of distributions. The book is hence of interest for companies operating not only in the area of data analytics, but also on logistics, energy and finance. It also concerns national statistical institutes and other institutions at European and international level, where microdata is aggregated to preserve confidentiality and allow for analysis at the appropriate regional level. Academics will find in the analysis of distributional data a challenging up-to-date field of research.

This book provides an introduction to spatial analyses concerning disaggregated (or micro) spatial data. Particular emphasis is put on spatial data compilation and the structuring of the connections between the observations. Descriptive analysis methods of spatial data are presented in order to identify and measure the spatial, global and local dependency. The authors then focus on autoregressive spatial models, to control the problem of spatial dependency between the residues of a basic linear statistical model, thereby contravening one of the basic hypotheses of the ordinary least squares approach. This book is a popularized reference for students looking to work with spatialized data, but who do not have the advanced statistical theoretical basics.

Generate and Analyze Multi-Level Data Spatial microsimulation involves the generation, analysis, and modeling of individual-level data allocated to geographical zones. Spatial Microsimulation with R is the first practical book to illustrate this approach in a modern statistical programming language. Get Insight into Complex Behaviors The book progresses from the principles underlying population synthesis toward more complex issues such as household allocation and using the results of spatial microsimulation for agent-based modeling. This equips you with the skills needed to apply the techniques to real-world situations. The book demonstrates methods for population synthesis by combining individual and geographically aggregated datasets using the recent R packages ipfp and mipfp. This approach represents the "best of both worlds" in terms of spatial resolution and person-level detail, overcoming issues of data confidentiality and reproducibility. Implement the Methods on Your Own Data Full of reproducible examples using code and data, the book is suitable for students and applied researchers in health, economics, transport, geography, and other fields that require individual-level data allocated to small geographic zones. By explaining how to use tools for modeling phenomena that vary over space, the book enhances your knowledge of complex systems and empowers you to provide evidence-based policy guidance.

The environment for obtaining information and providing statistical data for policy makers and the public has changed significantly in the past decade, raising questions about the fundamental survey paradigm that underlies federal statistics. New data sources provide opportunities to develop a new paradigm that can improve timeliness, geographic or subpopulation detail, and statistical efficiency. It also has the potential to reduce the costs of producing federal statistics. The panel's first report described federal statistical agencies' current paradigm, which relies heavily on sample surveys for producing national statistics, and challenges agencies are facing; the legal frameworks and mechanisms for protecting the privacy and confidentiality of statistical data and for providing researchers access to data, and challenges to those frameworks and mechanisms; and statistical agencies access to alternative sources of data. The panel recommended a new approach for federal statistical programs that would combine diverse data sources from government and private sector sources and the creation of a new entity that would provide the foundational elements needed for this new approach, including legal authority to access data and protect privacy. This second of the panel's two reports builds on the analysis, conclusions, and recommendations in the first one. This report assesses alternative methods for implementing a new approach that would combine diverse data sources from government and private sector sources, including describing statistical models for combining data from multiple sources; examining statistical and computer science approaches that foster privacy protections; evaluating frameworks for assessing the quality and utility of alternative data sources; and various models for implementing the recommended new entity. Together, the two reports offer ideas and recommendations to help federal statistical agencies examine and evaluate data from alternative sources and then combine them as appropriate to provide the country with more timely, actionable, and useful information for policy makers, businesses, and individuals.

Using data from several countries, including Cote d'Ivoire, India, Pakistan, Taiwan, and Thailand, this book analyzes household survey data from developing countries and illustrates how such data can be used to cast light on a range of short-term and long-term policy issues.