|Evaluation of National Assessment of Educational Progress Achievement Levels
For over 40 years, the National Assessment of Educational Progress (NAEP) has provided the only continuing measure of what America’s students know and can do in various subject areas. Over the past fifteen years, the NRC’s Board on Testing and Assessment has evaluated various aspects of NAEP and its design and reporting for the Department of Education.
This study focuses on the NAEP achievement levels which are used to provide information that can be easily understood and used by policy makers, educators, and the public. NAEP uses three achievement levels – basic, proficient, and advanced – and reports the percentage of test takers who score at each level. NAEP also reports the percentage of test takers who score below the basic level. Is this way of reporting test performance informative to the public? Is it done in a way that produces results that are reliable, valid, and reasonable? An NRC study committee will be appointed under BOTA to answer these questions and will begin its work in early 2015.
To meet its charge, the committee will:
• Review the procedures that the National Assessment Governing Board and the National Center for Education Statistics use to determine (or “set”) the achievement levels and the minimum score needed to be placed in each category.
• Synthesize the relevant research literature on setting achievement levels.
• Examine the ways that the achievement level reports are used, the interpretations made of them, and the validity of those interpretations.
The committee’s final report, expected in spring of 2016, will describe its findings about the achievement levels and how the levels are used in NAEP reporting. If warranted, the committee's report will also provide recommendations about ways that the setting and use of achievement levels for NAEP can be improved.
Sponsor: U.S. Department of Education
Background about NAEP:
Assessments are conducted periodically in mathematics, reading, science, writing, the arts, civics, economics, geography, U.S. history, and beginning in 2014, in Technology and Engineering Literacy. Each subject is assessed at grades 4, 8, and 12, although not all grades are assessed each time. NAEP does not report results for individual students. Instead, it is given to representative samples of students so that results can be reported for the nation, states, and some urban districts.
Christopher Edley, Jr. (Chair), William H. Orrick, Jr. Distinguished Professor, University of California, Berkeley School of Law
Peter Afflerbach, Professor of Reading, Department of Teaching and Learning, Policy and Leadership, University of Maryland
Sybilla Beckmann, Josiah Meigs Distinguished Teaching Professor of Mathematics, University of Georgia
Russell Bernard, Director, Institute for Social Science Research, Arizona State University, and Professor Emeritus, University of Florida
Karla Egan, Associate, National Center for the Improvement of Educational Assessment
David J. Francis, Hugh Roy and Lillie Cranz Cullen Distinguished Professor and Chair, Department of Psychology, University of Houston
Margaret E. Goertz, Professor Emerita, Graduate School of Education, University of Pennsylvania
Laura Hamilton, Senior Behavioral Scientist, Associate Director, RAND Education, RAND Corporation
Brian Junker, Professor, Carnegie Mellon University
Suzanne Lane, Professor, School of Education, University of Pittsburgh
Sharon J. Lewis, Council of Great City Schools, Retired
Bernard L. Madison, Professor, University of Arkansas
Scott Norton, Strategic Initiative Director of Standards, Assessment, and Accountability, Council of Chief State School Officers
Sharon Vaughn, H.E. Hartfelder/Southland Corp. Regents Chair, University of Texas at Austin
Lauress Wise, Principal Scientist, Human Resources Research Organization
Judith Koenig, Study Director
Natalie Nielsen, Acting Board Director
Kelly Arrington, Senior Program Assistant
First Meeting- February 19-20, 2015
Second meeting and Workshop- May 27-28, 2015
Workshop on Interpretations and Uses of NAEP Achievement Levels: May 27, 2015
Previous BOTA Reports Related to NAEP