"Nonparametric regression models continue to receive more attention and appreciation with the advance in both statistical methodology and computing software over the last three decades. These methods use smooth, flexible functional forms of the predictor to describe the dependency of the mean of responses on a set of covariates. The shape of the smooth curve is directly estimated from the data. While several competing approaches are available for such modelling, penalized splines (P-splines) are a powerful and applicable smoothing technique that restricts the influence of knots in regression splines. P-splines can be viewed as a particular case of generalized linear mixed models (GLMMs). To achieve a smooth function, we can use the GLMM to shrink the regression coefficients of knot points from a regression spline towards zero, by including them as random effects. The resulting models are referred to as semiparametric mixed models (SPMMs). The main advantage of this approach is that the smoothing parameter, which controls the trade-off between bias and variance, may be directly estimated from the data. Moreover, we can take full advantage of existing methods and software for GLMMs. This thesis addresses several unresolved methodological issues related to the implementation of SPMMs, especially for binary outcomes. First, how best to estimate flexible regression curves when the outcomes are correlated and binary is unclear, especially when cluster sizes are small and also when the validity of the model assumptions are violated. In this regard, in the first manuscript, I compare the performance of the likelihood-based and Bayesian approaches to estimate SPMMs for correlated binary data. I also investigate the effect of concurvity (analogous to multicollinearity in linear regression) among covariates on estimates of SPMMs components, an issue that has not yet been studied in the SPMMs context. Next, while it is evident that SPMMs performed very well in recapturing the true curves, it remained unclear how curve fitting via SPMMs impacts the estimation of correlation and variance parameters in complicated data situations arising from, for example, longitudinal studies where data are both over-dispersed and serially correlated. In the second manuscript, I extend the SPMM for analyzing over-dispersed and serially correlated longitudinal data and systematically evaluate the effect of smoothing using SPMMs on the correlation and variance parameter estimates. I also compare the performance of SPMMs to other simpler approaches for estimating the nonlinear association such as fractional polynomials, and quadratic polynomial. Finally, in the third manuscript, I introduce a novel LASSO type penalized splines in the SPMM setting to investigate if the curve fitting performance can be improved using a LASSO type absolute value penalty (to the changes in fit at knots) rather than using typical ridge regression penalty. All these methods are also applied to different real-life data sets." --

Now in widespread use, generalized additive models (GAMs) have evolved into a standard statistical methodology of considerable flexibility. While Hastie and Tibshirani's outstanding 1990 research monograph on GAMs is largely responsible for this, there has been a long-standing need for an accessible introductory treatment of the subject that also emphasizes recent penalized regression spline approaches to GAMs and the mixed model extensions of these models. Generalized Additive Models: An Introduction with R imparts a thorough understanding of the theory and practical applications of GAMs and related advanced models, enabling informed use of these very flexible tools. The author bases his approach on a framework of penalized regression splines, and builds a well-grounded foundation through motivating chapters on linear and generalized linear models. While firmly focused on the practical aspects of GAMs, discussions include fairly full explanations of the theory underlying the methods. Use of the freely available R software helps explain the theory and illustrates the practicalities of linear, generalized linear, and generalized additive models, as well as their mixed effect extensions. The treatment is rich with practical examples, and it includes an entire chapter on the analysis of real data sets using R and the author's add-on package mgcv. Each chapter includes exercises, for which complete solutions are provided in an appendix. Concise, comprehensive, and essentially self-contained, Generalized Additive Models: An Introduction with R prepares readers with the practical skills and the theoretical background needed to use and understand GAMs and to move on to other GAM-related methods and models, such as SS-ANOVA, P-splines, backfitting and Bayesian approaches to smoothing and additive modelling.

Although standard mixed effects models are useful in a range of studies, other approaches must often be used in correlation with them when studying complex or incomplete data. Mixed Effects Models for Complex Data discusses commonly used mixed effects models and presents appropriate approaches to address dropouts, missing data, measurement errors, censoring, and outliers. For each class of mixed effects model, the author reviews the corresponding class of regression model for cross-sectional data. An overview of general models and methods, along with motivating examples After presenting real data examples and outlining general approaches to the analysis of longitudinal/clustered data and incomplete data, the book introduces linear mixed effects (LME) models, generalized linear mixed models (GLMMs), nonlinear mixed effects (NLME) models, and semiparametric and nonparametric mixed effects models. It also includes general approaches for the analysis of complex data with missing values, measurement errors, censoring, and outliers. Self-contained coverage of specific topics Subsequent chapters delve more deeply into missing data problems, covariate measurement errors, and censored responses in mixed effects models. Focusing on incomplete data, the book also covers survival and frailty models, joint models of survival and longitudinal data, robust methods for mixed effects models, marginal generalized estimating equation (GEE) models for longitudinal or clustered data, and Bayesian methods for mixed effects models. Background material In the appendix, the author provides background information, such as likelihood theory, the Gibbs sampler, rejection and importance sampling methods, numerical integration methods, optimization methods, bootstrap, and matrix algebra. Failure to properly address missing data, measurement errors, and other issues in statistical analyses can lead to severely biased or misleading results. This book explores the biases that arise when naïve methods are used and shows which approaches should be used to achieve accurate results in longitudinal data analysis.

The linear mixed model has become the main parametric tool for the analysis of continuous longitudinal data, as the authors discussed in their 2000 book. Without putting too much emphasis on software, the book shows how the different approaches can be implemented within the SAS software package. The authors received the American Statistical Association's Excellence in Continuing Education Award based on short courses on longitudinal and incomplete data at the Joint Statistical Meetings of 2002 and 2004.

Linear mixed models are a powerful inferential tool in modern statistics and have a wide range of applications. Recent advances utilize the connection between penalized spline smoothing and mixed models for efficient implementation of nonparametric and semiparametric regression techniques. These become increasingly important to adequately model the relationship between response variables and covariates. However, despite their common use, some open questions regarding the inference in mixed models still remain. This dissertation is aimed at improving the methodology for inference on random effects. An important special case is testing for polynomial regression against a general smooth alternative modeled by mixed model penalized splines. Our motivating application is the assessment of non-linearity for air pollution dose-response functions in the epidemiological Airgene study. Testing for a zero random effects variance is a non-standard testing problem. First, the tested parameter is on the boundary of the parameter space under the null hypothesis. Second, in linear mixed models observations are generally not independent. While in longitudinal linear mixed models there are at least independent subjects or units, such a subdivision of the data is not possible for mixed model penalized spline smoothing. We first investigate the asymptotic distribution of the restricted likelihood ratio test statistic when testing for polynomial regression using mixed model penalized splines. We show that asymptotic results on boundary testing for independent observations do not hold here. This is due to the asymptotic non-normality of the score statistic. Fundamentally, this is caused by the dependence of observations induced by penalized splines. We find that this dependence cannot be ignored, as it is inherently necessary for the attainment of smooth curves. Different approaches to this testing problem are therefore necessary. Subsequently, we provide finite sample alternatives for testing for zero random effect variances in linear mixed models. The class of models we consider is more general than has previously been covered, including models with moderate numbers of clusters, unbalanced designs, or nonparametric smoothing. We also allow more than one random effect in the model. We propose two approximations to the finite sample null distribution of the restricted likelihood ratio test statistic. Extensive simulations show that both outperform the chi-square mixture approximation and parametric bootstrap currently used, as well as several F-type tests. Finally, we discuss model selection for mixed model penalized splines using the Akaike Information Criterion. The criterion based on the marginal likelihood is found not to be asymptotically unbiased for the expected relative Kullback-Leibler distance. In fact, it is biased towards the simpler model. An alternative is provided using our results on restricted likelihood ratio testing.

This book describes how generalised linear modelling procedures can be used in many different fields, without becoming entangled in problems of statistical inference. The author shows the unity of many of the commonly used models and provides readers with a taste of many different areas, such as survival models, time series, and spatial analysis, and of their unity. As such, this book will appeal to applied statisticians and to scientists having a basic grounding in modern statistics. With many exercises at the end of each chapter, it will equally constitute an excellent text for teaching applied statistics students and non- statistics majors. The reader is assumed to have knowledge of basic statistical principles, whether from a Bayesian, frequentist, or direct likelihood point of view, being familiar at least with the analysis of the simpler normal linear models, regression and ANOVA.

Mixed-effects models have found broad applications in various fields. As a result, the interest in learning and using these models is rapidly growing. On the other hand, some of these models, such as the linear mixed models and generalized linear mixed models, are highly parametric, involving distributional assumptions that may not be satisfied in real-life problems. Therefore, it is important, from a practical standpoint, that the methods of inference about these models are robust to violation of model assumptions. Fortunately, there is a full scale of methods currently available that are robust in certain aspects. Learning about these methods is essential for the practice of mixed-effects models.This research monograph provides a comprehensive account of methods of mixed model analysis that are robust in various aspects, such as to violation of model assumptions, or to outliers. It is suitable as a reference book for a practitioner who uses the mixed-effects models, and a researcher who studies these models. It can also be treated as a graduate text for a course on mixed-effects models and their applications.

This user guide presents a popular smoothing tool with practical applications in machine learning, engineering, and statistics.