Skip to Main Content
The National Academies of Sciences, Engineering and Medicine
Committee on Law and Justice
Division of Behavioral and Social Sciences and Education




 Seminar on Evaluation

April 13, 2010


The National Academies

Keck Building, Room 201
500 5th Street, NW, Washington, DC



At the request of the National Institute of Justice, the Committee on Law and Justice conducted a seminar on program evaluation, focusing especially on issues that are relevant to government organizations such as NIJ, which deal with both research and practice. Philip Cook, Charles Manski, and David Weisburd organized this seminar.

Workshop Moderator – Dr. Philip Cook

Tuesday, April 13, 2010, 11:15 a.m. - 4:30 p.m.


Informing Policy 

Dr. Charles Manski, Discussion Leader
Dr. Wesley Skogan, Discussant
Dr. Barbara Devaney, Discussant


This session will address when and how to assess various treatments and policies. It would cover the types of questions you’d want to answer about any proposed treatment or policy in the justice area, such as the costs and benefits of the policy, its implementation, and its impact. Also, who is the audience that wants the answers to these questions?


Types of Evaluation Methodologies 

Dr. Philip Cook, Discussion Leader
Dr. Adele Harrell, Discussant


While the first session will focus on a broad range of questions and discuss the appropriate methodologies to answer them, this second session will focus on a more detailed discussion of particular evaluation methodologies, especially those that seem relevant to the issues dealt with by NIJ. The issue of resource and time requirements related to particular methodologies may be addressed, as well as the potential audiences for these evaluations.

NIJ Infrastructure 

Dr. David Weisburd, Discussion Leader


This session will focus on how NIJ can and should strengthen its infrastructure and internal research in order to support the development of evaluation programs and the funding of evaluation projects in the future. There are three key concerns that will be addressed in this session. The first focuses on how NIJ can take a more strategic approach to the problem of evaluation. Should NIJ simply be responding to requests from other agencies regarding evaluation priorities or should NIJ be more strategic in deciding upon long-term evaluation problems and projects? The second relates to how NIJ should assess and evaluate the quality and appropriateness of proposals in this area. Should NIJ be developing an in house staff capable of assessing study quality and the reasonableness of successful outcomes? Or should NIJ be looking to outside consultants to fill such roles? The final concern relates to the responsibilities of NIJ in advancing methodology in evaluation research in crime and justice. Should NIJ take a proactive role, for example in funding an evaluation research methods program, or should it leave this responsibility to others? What internal capacity should it build in order to have a stronger evaluation component and what would be the costs related to various levels of capacity?


Dr. Philip Cook, Moderator








Seminar on Evaluation 

Speakers Biographical Data


Barbara Devaney, an economist, is Senior Vice President of Mathematica Policy Research and Managing Director of Human Services Research. Throughout her 30-year career, she has worked on numerous studies to design and implement rigorous evaluations of a broad range of substantive topic areas. These evaluation designs address key issues—issues such as the need for a random assignment or strong quasi-experimental design, the importance of a comprehensive implementation analysis, the development of a conceptual model for how interventions can change key outcomes, sample design and power analyses, and analysis and reporting plans. She is currently the Co-Director of Building Strong Families, a large demonstration and evaluation of well-designed interventions to strengthen couple relationships. She was Co-Principal Investigator of the evaluation of abstinence education programs, which involved a random assignment evaluation of abstinence only education programs funded under welfare reform. She has studied extensively food and nutrition assistance programs, including the Supplemental Nutrition Assistance Program (formerly the Food Stamp Program), the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), and the school meals programs. Dr. Devaney received her Ph.D. in economics from the University of Michigan and also held positions in the Terry Sanford School of Public Policy at Duke University and the Department of Economics at Johns Hopkins University. She served on three subcommittees of the Food and Nutrition Board of the Institute of Medicine: (1) the Committee to Review the WIC Food Packages; (2) the Subcommittee on Interpretation and Uses of Dietary Reference Intakes; and (3) the Committee on Scientific Evaluation of WIC Nutrition Risk Criteria.


Thomas E. Feucht is Executive Senior Science Advisor at the National Institute of Justice (NIJ), U.S. Department of Justice. He has been a member of the federal government’s Senior Executive Service since 2005 and has been with the National Institute of Justice since 1994. Dr. Feucht received his doctorate in sociology in 1986 from the University of North Carolina-Chapel Hill with an emphasis on quantitative research methods and statistics. From 1987 to 1994, Dr. Feucht served on the faculty at Cleveland State University (CSU) in the Sociology Department and the College of Urban Affairs. Dr. Feucht joined NIJ in 1994. Starting in 1996, he served as Chief of the Crime Control and Prevention Division in NIJ’s Office of Research and Evaluation (ORE) until he became ORE’s deputy director (1998) and later, its director (2002). Dr. Feucht serves on the Social, Behavioral, and Economic Sciences (SBE) Subcommittee of the National Science and Technology Council Committee on Science, of the White House Office of Science and Technology Policy. From 1998 to 2000, Dr. Feucht served as chief of staff to the Attorney General’s Methamphetamine Interagency Task Force, established as part of the 1996 Methamphetamine Control Act. He has conducted and published research in the areas of policing and terrorism, substance abuse, intravenous drug use and HIV, prostitution, prison drug use, and school violence.

Naomi Goldstein is Director of the Office of Planning, Research and Evaluation (OPRE) in the Administration for Children and Families (ACF) at the Department of Health and Human Services. She is responsible for advising the Assistant Secretary for Children and Families on increasing the effectiveness and efficiency of ACF programs. Prior to her appointment as Director of OPRE in November, 2004, Ms. Goldstein served as Director of the Division of Child and Family Development in OPRE. Previously she directed the United States Postal Service Commission on A Safe and Secure Workplace, an independent commission that examined workplace violence affecting the Postal Service and the nation. She served as Project Manager for the Urban Institute’s Assessing the New Federalism project, and as Executive Officer in the Office of the Assistant Secretary for Planning and Evaluation at HHS. Earlier in her career, she served in the Massachusetts state government and developed infant mortality prevention programs at Brigham and Women’s Hospital in Boston. Ms. Goldstein received a B.A. in Philosophy from Yale University, a Masters in Public Policy from the Kennedy School of Government, and a Ph.D. in Public Policy from Harvard University.

Adele V. Harrell is the retired founding Director of the Justice Policy Center at the Urban Institute. Her research over the past 30 years has focused on criminal justice responses to drug abuse and domestic violence. She served as Codirector of the Evaluation of the Judicial Oversight Demonstration, a multi-site longitudinal evaluation of court and community services to reduce intimate partner violence. Other large scale experimental and quasi-experimental studies she has directed include evaluations of Children at Risk, a community-based drug prevention program, Breaking the Cycle Demonstration, a system-wide intervention for drug involved felony defendants, the DC Superior Court’s Drug Intervention Program, and the Brooklyn Treatment Court, drug court services for female offenders. She is a Fellow of the Academy of Experimental Criminology and a member of the National Advisory Panel for the Office of National Drug Control Policy. She has a Ph.D. in Sociology from George Washington University.


Kristina Rose is the Acting Director of the National Institute of Justice (NIJ), where she oversees the research, development, and evaluation activities of the Department of Justice. She is also responsible for leading agency-wide special projects and initiatives that cut across both social and physical sciences. Prior to her role as Acting Director, she served as the Senior Advisor to the Director of NIJ, providing expert advice, guidance, and assistance on criminal justice policy and management issues. From 2001 to 2006, Ms. Rose served as the Chief of Staff at the Department’s Office on Violence Against Women. As Chief of Staff, Ms. Rose was responsible for assisting with the overall operation and internal coordination of the office as well as serving as an Advisor to the Director on policy and administrative issues. Ms. Rose prepared all speeches and other specially written materials for the Director and other Department of Justice officials on domestic violence, sexual assault, and stalking. Ms. Rose has a B.S. in sociology from George Mason University and an M.S. in criminal justice from Northeastern University.


Ellen Scrivner is Deputy Director of the National Institute of Justice. Scrivner is a nationally recognized expert on a broad range of policing issues. She started her career as a psychologist for two local police departments in the Washington, D.C., metropolitan area and later served as a deputy superintendent of the Chicago Police Department. Scrivner was Deputy Director of the Community Oriented Policing Services Office at the U.S. Department of Justice and has served as a consultant to the Office of Law Enforcement Coordination at the FBI. She was a visiting scholar at NIJ from 1992 to 1994. She was director of the Leadership Academy at John Jay College of Criminal Justice in New York City.


Wesley G. Skogan is Professor of Political Science Department and the Institute for Policy Research at Northwestern University. His research focuses on the interface between the public and the legal system, particularly community-oriented policing. His most recent book is Police and Community in Chicago (2006). He is the author of two lengthy reports in the Home Office Research Series examining citizen contact and satisfaction with policing in Britain, and the co-editor of a policy-oriented report from the National Research Council in Washington, DC: Fairness and Effectiveness in Policing: The Evidence. Another line of his research concerns neighborhood and community responses to crime. This includes work on fear of crime, the impact of crime on neighborhood life, and crime prevention efforts by community organizations. Prof. Skogan has also been involved in research on criminal victimization and the evaluation of service programs for victims.


Christopher Trenholm is a Senior Economist and Associate Director for Health Research at Mathematica Policy Research. He is an expert in risk reduction programs for youth, maternal and child health and health insurance programs. Currently, Dr. Trenholm co-directs a major impact evaluation of adolescent pregnancy prevention programs, which included a recent review of the evidence-based behind various interventions designed to prevent teen pregnancies and births. This current project follows a widely publicized Mathematica evaluation examining the impact of abstinence education programs, which Dr. Trenholm likewise co-directed. Dr. Trenholm has also served as principal investigator for several recent studies of efforts to expand health insurance coverage to children, including evaluations of the Robert Wood Johnson Foundation’s Covering Kids and Families program and Maximizing Enrollment for Kids program, and a congressionally-mandated evaluation of the Children’s Health Insurance Program (CHIP).


Phelan Wyrick is a Senior Social Science Analyst at the National Institute of Justice, U.S. Department of Justice. Dr. Wyrick's work in gang reduction has recently received the prestigious Attorney General's Award for Outstanding Contributions to Community Partnerships for Public Safety. Dr. Wyrick’s work for the Attorney General's Anti-Gang Initiative stemmed from his earlier work implementing and developing community-based anti-gang programs for the Office of Juvenile Justice and Delinquency Prevention's Gang Reduction Program.


Home BOHSI Home BOSE Home BECS Home BCYF Home TAC Home Home BOTA Home CPOP Home CNSTAT Home CLAJ Home BBCSS Home Home About CLAJ CLAJ Mission CLAJ Members CLAJ Staff Other Sites of Interest About BOSE BOSE Mission BOSE Members BOSE Staff Our Sponsors Visiting our Buildings About CPOP CPOP Mission CPOP Members CPOP Staff About BBCSS BBCSS History BBCSS Members BBCSS Staff Our Sponsors Forum Home About CNSTAT CNSTAT Members CNSTAT Staff CNSTAT Mission Other Sites of Interest TEstTAc2 About BCYF BCYF History BCYF Graphic Illustration BCYF Members BCYF Staff About BOHSI BOHSI History BOHSI Members BOHSI Staff Our Sponsors ABOUT DBASSE Mission Advisory Committee Hauser Policy Impact Fund DBASSE Staff DBASSE Brochure Visiting our Buildings About Us Mission and Goals History Members Sponsors About BOTA BOTA Mission BOTA Members Visiting Our Buildings