Evaluation of scientific research, particularly of research which is supported by government funds, is a matter of growing concern in virtually every nation. It is no longer adequate to expect that the value of investments in research will be judged in long-term historical perspective. Resources are scarce and policy-makers are looking for ways to assure that these resources are used in the most effective way. From the life-or-death evaluations of academic research institutes in the post-communist countries to the Government Performance and Results Act(GPRA) in the United States, research evaluation has become a topic of utmost importance in science policy. Evaluation often has substantial consequences for researchers and research institutions, including restructuring, shifting of priorities, budget reductions, or evenclosures. Therefore it is essential that evaluation is done systematically and objectively, with methodologies that can be understood and trusted by those concerned. This book is based on a NATO Advanced Research Workshop, co-organized by the Academy of Sciences of the Czech Republic and the American Association for the Advancement of Science. It describes a range of the most up-to-date methods of science evaluation and the experience with their implementation in many countries. This book can be of interest to researchers, policy-makers, practitioners of science evaluation and many others interested in science policy.

The past half-century has witnessed a dramatic increase in the scale and complexity of scientific research. The growing scale of science has been accompanied by a shift toward collaborative research, referred to as "team science." Scientific research is increasingly conducted by small teams and larger groups rather than individual investigators, but the challenges of collaboration can slow these teams' progress in achieving their scientific goals. How does a team-based approach work, and how can universities and research institutions support teams? Enhancing the Effectiveness of Team Science synthesizes and integrates the available research to provide guidance on assembling the science team; leadership, education and professional development for science teams and groups. It also examines institutional and organizational structures and policies to support science teams and identifies areas where further research is needed to help science teams and groups achieve their scientific and translational goals. This report offers major public policy recommendations for science research agencies and policymakers, as well as recommendations for individual scientists, disciplinary associations, and research universities. Enhancing the Effectiveness of Team Science will be of interest to university research administrators, team science leaders, science faculty, and graduate and postdoctoral students.

Evaluation researchers are tasked with providing the evidence to guide programme building and to assess its outcomes. As such, they labour under the highest expectations - bringing independence and objectivity to policy making. They face huge challenges, given the complexity of modern interventions and the politicised backdrop to all of their investigations. They have responded with a huge portfolio of research techniques and, through their professional associations, have set up schemes to establish standards for evaluative inquiry and to accredit evaluation practitioners. A big question remains. Has this monumental effort produced a progressive, cumulative and authoritative body of knowledge that we might think of as evaluation science? This is the question addressed by Ray Pawson in this sequel to Realistic Evaluation and Evidence-based Policy. In answer, he provides a detailed blueprint for an evaluation science based on realist principles.

Economists, decision analysts, management scientists, and others have long argued that government should take a more scientific approach to decision making. Pointing to various theories for prescribing and rational izing choices, they have maintained that social goals could be achieved more effectively and at lower costs if government decisions were routinely subjected to analysis. Now, government policy makers are putting decision science to the test. Recent government actions encourage and in some cases require government decisions to be evaluated using formally defined principles 01' rationality. Will decision science pass tbis test? The answer depends on whether analysts can quickly and successfully translate their theories into practical approaches and whether these approaches promote the solution of the complex, highly uncertain, and politically sensitive problems that are of greatest concern to government decision makers. The future of decision science, perhaps even the nation's well-being, depends on the outcome. A major difficulty for the analysts who are being called upon by government to apply decision-aiding approaches is that decision science has not yet evolved a universally accepted methodology for analyzing social decisions involving risk. Numerous approaches have been proposed, including variations of cost-benefit analysis, decision analysis, and applied social welfare theory. Each of these, however, has its limitations and deficiencies and none has a proven track record for application to govern ment decisions involving risk. Cost-benefit approaches have been exten sively applied by the government, but most applications have been for decisions that were largely risk-free.

With insightful discussion of program evaluation and the efforts of the Centers for Disease Control, this book presents a set of clear-cut recommendations to help ensure that the substantial resources devoted to the fight against AIDS will be used most effectively. This expanded edition of Evaluating AIDS Prevention Programs covers evaluation strategies and outcome measurements, including a realistic review of the factors that make evaluation of AIDS programs particularly difficult. Randomized field experiments are examined, focusing on the use of alternative treatments rather than placebo controls. The book also reviews nonexperimental techniques, including a critical examination of evaluation methods that are observational rather than experimentalâ€"a necessity when randomized experiments are infeasible.

This book is written for members of the scholarly research community, and for persons involved in research evaluation and research policy. More specifically, it is directed towards the following four main groups of readers: – All scientists and scholars who have been or will be subjected to a quantitative assessment of research performance using citation analysis. – Research policy makers and managers who wish to become conversant with the basic features of citation analysis, and about its potentialities and limitations. – Members of peer review committees and other evaluators, who consider the use of citation analysis as a tool in their assessments. – Practitioners and students in the field of quantitative science and technology studies, informetrics, and library and information science. Citation analysis involves the construction and application of a series of indicators of the ‘impact’, ‘influence’ or ‘quality’ of scholarly work, derived from citation data, i.e. data on references cited in footnotes or bibliographies of scholarly research publications. Such indicators are applied both in the study of scholarly communication and in the assessment of research performance. The term ‘scholarly’ comprises all domains of science and scholarship, including not only those fields that are normally denoted as science – the natural and life sciences, mathematical and technical sciences – but also social sciences and humanities.

This book constitutes the refereed proceedings of the 7th International Conference on Design Science Research in Information Systems and Technology, DERIST 2012, held in Las Vegas, NV, USA, in May 2012. The 24 revised full papers presented together with 7 revised short papers were carefully reviewed and selected from 44 submissions. The papers are organized in topical sections on DSRIS in practice, DSRIS methodologies and techniques, social and environmental aspects of DSRIS, theory and theory building in DSRIS, and evaluation of DSRIS projects.

Program Theory-Driven Evaluation Science fills the gap between 21st century literature on evaluation and what is happening in practice. It features detailed examples of how evaluations actually unfold in practice to develop people, programs, and organizations. Commonly accepted strategies for practicing evaluation are outlined, followed by comprehe

This important new work provides a comprehensive discussion of the customer satisfaction evaluation problem. It presents an overview of the existing methodologies as well as the development and implementation of an original multicriteria method dubbed MUSA.

This text provides an introduction to the theory and practice of internal evaluation. It presents the stages of internal evaluation growth, ways of identifying users' needs and selecting appropriate evaluation methods.