External Evaluation of NIDRR and its Grantees
|
Project Scope
An ad hoc committee will conduct an evaluation of National Institute on Disability and Rehabilitation Research (NIDRR)-funded grants. The committee will develop an overall framework and evaluation design that will be used to review NIDRR’s program priority and grant review and final selection process (process evaluation) and review grantee outputs for a representative sample of grantees in three areas to be specified by the committee that are representative of the NIDRR portfolio, e.g., medical, engineering, and social science research(grantee output or summative evaluation).
In developing the overall framework and evaluation design, the committee will review and assess the grant selection process used by NIDRR in awarding grants; develop a conceptual framework for a review of outputs produced by NIDRR grantees and an evaluation design aimed at assessing the quality of the these outputs; develop a sampling plan for selecting a representative sample of NIDRR grantees in three major areas representative of NIDRR's research portfolio; and develop criteria for assessing the quality of four specific types of grantee outputs (publications; tools, measures, and intervention protocols; technology products and devices; and informational products). The criteria developed by the committee will be applied to the selected grantee outputs in each of the three major areas of the NIDRR portfolio specified by the committee.
The questions to be addressed by the evaluation will be informed by OMB’s 2008 PART guidance, with a focus on quality. Specific key questions to be addressed by the committee include:
Process Evaluation:
-To what extent is NIDRR's grant selection process conducted in such as way as to enhance the quality of the final results?
-To what extent are the peer reviews of grant applications done in such as way to enhance the quality of the final results?
-What planning and budgetary processes does the grantee use to promote high quality outputs?
Summative Evaluation of Grantee Outputs:
-To what extent are the final outputs from NIDRR grants of high quality?
-To what extent are the results of the reviewed research and development outcomes used to inform new projects by both the grantee and NIDRR?
The committee will prepare a letter report with its assessment of the evaluation design and evaluation process and recommendations on design changes for future evaluations. A final report will subsequently be produced that describes and assesses the NIDRR priority setting and grant selection processes (process evaluation); describes the grantee output review process developed by the committee, including the review criteria used and key research areas specified; assesses the application of the review process; summarizes the results of the output reviews that represent the three major areas of NIDRR's portfolio; assesses the review process and recommends any needed revisions, including to the output review criteria; and makes recommendations for future output reviews (summative evaluation).
The project is sponsored by the U.S. Department of Education.
The approximate start date for the project is 9/25/09.
The letter report is expected to be issued in spring 2011. The final report will be issued in fall 2011.
Members |
| Dr. David H. Wegman (Chair) - University of Massachusetts |
| Dr. Thomas J. Armstrong - University of Michigan |
| Dr. Burt S. Barnow - Johns Hopkins University |
| Dr. Leighton Chan - National Institutes of Health |
| Dr. Peter C. Esselman - Harborview Medical Center |
| Dr. Walter R. Frontera - University of Puerto Rico School of Medicine |
| Dr. Glenn T. Fujiura - University of Illinois at Chicago |
| Dr. Bruce M. Gans - Kessler Medical Rehabilitation Research and Education Corporation |
| Dr. Ian Graham - Canadian Institutes of Health Research |
| Dr. Lisa I. Iezzoni - Massachusetts General Hospital |
| Dr. Alan M. Jette - Boston University School of Public Health |
| Dr. Thubi Kolobe - University of Oklahoma Health Science Center |
| Dr. Pamela Loprest - The Urban Institute |
| Dr. Kathryn E. Newcomer - The George Washington University |
| Ms. Patricia M. Owens - Government Accountability Office |
| Dr. Robert G. Radwin - University of Wisconsin-Madison |
| For more information, see the complete record at the National Academies' Current Project site |
