Wednesday, July 23, 2014
Laboratory Assessments Board The National Academies
National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council
LAB Home
About LAB
Rosters
Current Projects
LAB Reports
Staff and Contact Information
DEPS Home

LAB Reports
 

Past reports of the former Board on Assessment of NIST Programs are online at the links below - a limited number of hard copies are available by contacting the NRC at (202) 334-3311.

LAB brochure (PDF)

  

                                         

2013-2014 Assessment of the Army Research Laboratory: Interim Report (2014)

The National Research Council's Army Research Laboratory Technical Assessment Board provides biennial assessments of the scientific and technical quality of the research, development, and analysis programs at the Army Research Laboratory, focusing on ballistics sciences, human sciences, information sciences, materials sciences, and mechanical sciences.

This interim report summarizes the findings of the Board for the first year of this biennial assessment. During the first year the Board examined the following elements: within ballistic sciences, terminal ballistics; within human sciences, translational neuroscience and soldier simulation and training technology; within information sciences, autonomous systems; and within materials sciences, energy materials and devices, photonic materials and devices, and biomaterials. The review of autonomous systems included examination of the mechanical sciences competency area for autonomous systems. A second, final report will subsume the findings of this interim report and add the findings from the second year of the review, during which the Board will examine additional elements.

                                       

 

An Assessment of the National Institute of Standards and Technology Center for Neutron Research:

Fiscal Year 2013 (2013)  

The National Institute of Standards and Technology (NIST) Center for Neutron Research (NCNR), with its strong tradition of hiring and developing excellent scientific and technical staff, is one of the leading institutions worldwide in neutron  v instrumentation, technology, and science. It is a very well-managed user facility. With the recent completion of a $95 million expansion, performed on time and on budget, it has enhanced its instrumentation capabilities and has constructed a new guide hall. This expansion further enhances NCNR s ability to meet high user demands a factor of two higher than capacity for experimentation to conduct cutting-edge research. NCNR's high scientific productivity is due, in part, to effective communication between the management and staff and with the internal and external user communities.

This report assesses the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now six, including two user facilities, as well as the adequacy of the laboratories resources. An Assessment of the National Institute of Standards and Technology Center for Neutron Research considers the merit of the current NCNR scientific and technical programs relative to current state-of-the-art programs; the degree to which the NCNR scientific and technical programs achieve their objectives and fulfill the mission of the NCNR; and the adequacy of the NCNR facilities, equipment, and human resources, as they affect the quality of the NCNR s scientific and technical programs.

 

 2013-assess-ar;

2011-2012 Assessment of the Army Research Laboratory
 

 

The charge of the Army Research Laboratory Technical Assessment Board (ARLTAB) is to provide biennial assessments of the scientific and technical quality of the research, development, and analysis programs at the Army Research Laboratory (ARL). The ARLTAB is assisted by six panels, each of which focuses on the portion of the ARL program conducted by one of ARL's six directorates1. When requested to do so by ARL, the ARLTAB also examines work that cuts across the directorates. For example, during 2011-2012, ARL requested that the ARLTAB examine crosscutting work in the areas of autonomous systems and network science.

The overall quality of ARL's technical staff and their work continues to be impressive. Staff continue to demonstrate clear, passionate mindfulness of the importance of transitioning technology to support immediate and longer-term Army needs. Their involvement with the wider scientific and engineering community continues to expand. Such continued involvement and collaboration are fundamentally important for ARL's scientific and technical activities and need to include the essential elements of peer review and interaction through publications and travel to attend professional meetings, including international professional meetings. In general, ARL is working very well within an appropriate research and development niche and has been demonstrating significant accomplishments, as exemplified in the following discussion, which also addresses opportunities and challenges.
 

                                     

Best Practices in Assessment of Research and Development Organizations—
Summary of a Workshop

The National Institute of Standards and Technology (NIST)--recognizing that information and insights gained through continual examination of practices for organizational assessment are useful for decision makers at organizations across the deferral, industrial, academic, and national laboratory sectors-recently requested that the National Research Council (NRC) organize a panel to review best practices in assessment of research and development (R&D) organizations. In response, the NRC established the Panel for Review of Best Practices in Assessment of Research and Development Organizations.

The panel was charged to consider means of assessing the following in a manner that satisfies the requirements of NIST to perform effective assessments but also identifies assessment methods that can be applied selectively to other R&D organizations. These methods include: technical merit and quality of the science and engineering work, the adequacy of the resources available to support high-quality work, the effectiveness of the agency's delivery of the services and products required to fulfill its goals, the degree to which the agency's current and planned R&D portfolio supports its mission, as well as the agency's flexibility to respond to changing economic, political, social and technological contexts.

As one means of data gathering, among others that the panel is performing toward development of a final report of its findings, the panel organized a planning committee for a workshop on best practices in assessment of R&D organizations. Best Practices in Assessment of Research and Development Organizations: Summary of a Workshop reviews the workshop conducted at the Keck Center of the National Academies in Washington, D.C., on March 19, 2012.

 

Best Practices in Assessment of Research and Development Organizations (Final Report)

Research and development (R&D) organizations are operated by government, business, academe, and independent institutes. The success of their parent organizations is closely tied to the success of these R&D organizations. In this report, organizations refers to an organization that performs research and/or development activities (often a laboratory), and parent refers to the superordinate organization of which the R&D organization is a part. When the organization under discussion is formally labeled a laboratory, it is referred to as such. The question arises: How does one know whether an organization and its programs are achieving excellence in the best interests of its parent? Does the organization have an appropriate research staff, facilities, and equipment? Is it doing the right things at high levels of quality, relevance, and timeliness? Does it lead to successful new concepts, products, or processes that support the interests of its parent?


This report offers assessment guidelines for senior management of organizations and of their parents. The report lists the major principles of assessment, noting that details will vary from one organization to another. It provides sufficient information to inform the design of assessments, but it does not prescribe precisely how to perform them, because different techniques are needed for different types of organizations.
 

Best Practices in Assessment of Research and Development Organizations covers three key factors that underpin the success of an R&D organization: (1) the mission of the organization and its alignment with that of the parents; (2) the relevance and impact of the organization's work; and (3) the resources provided to the organization, beginning with a high-quality staff and management.

 

A Review of the Manufacturing-related Programs at the National Institute of Standards and Technology:
Fiscal Year 2012

The mission of the National Institute of Standards and Technology (NIST) enables NIST to provide broad support for the advancement of U.S. manufacturing. Research and services supporting manufacturing are intended to be an important component in all of the NIST laboratories. Moreover, since manufacturing is a major part of the U.S. economy, the growth or loss of U.S. manufacturing jobs is a very important issue. Clearly, the successful execution of NIST's programs supporting manufacturing will have a significant impact on manufacturing jobs in the United States. With the multidisciplinary, multisector, and crosscutting nature of manufacturing, the Director of NIST requested that the National Research Council (NRC) assess the manufacturing-related programs at NIST in 2012.


Accordingly, a panel of experts was convened by the National Research Council to perform the assessment. The Panel on review of the Manufacturing-Related Programs at the national Institute of Standards and Technology visited the NIST campus in Gaithersburg, Maryland, on March 26-28, 2012. A Review of the Manufacturing-related Programs at the National Institute of Standards and Technology: Fiscal Year 2012 contains the results of the panel's assessment.


The assessment considered manufacturing research at NIST broadly, with emphasis on the specific advanced manufacturing areas: Nanomanufacturing (including Flexible Electronics); Smart Manufacturing (including Robotics); and Next-Generation Materials Measurements, Modeling, and Simulation. The area of Biomanufacturing also reviewed as a subset of the Nanomanufacturing review. As is to be expected for programs covering such wide scope, the boundaries among these broad areas are not rigid and there is some overlap among them. On the basis of its assessment, the panel formed the observations and recommendations which are detailed in this report.

 

2009-2010 Assessment of the Army

2009-2010 Assessment of hte Army Research Laboratory

The charge of the Army Research Laboratory Technical Assessment Board (ARLTAB) is to provide biannual assessments of the scientific and technical quality of the research, development, and analysis programs at the Army Research Laboratory (ARL). The advice provided in this report focuses on technical rather than programmatic considerations.

The Board is assisted by six National Research Council (NRC) panels, each of which focuses on the portion of the ARL program conducted by one of ARL's six directorates. When requested to do so by ARL, the Board also examines work that cuts across the directorates. The Board has been performing assessments of ARL since 1996. The current report summarizes its finding for the 2009-2010 period, during which 96 volunteer experts in fields of science and engineering participated in the following activities: visiting ARL annually, receiving formal presentations of technical work, examining facilities, engaging in technical discussions with ARL staff, and reviewing ARL technical materials.

The Board continues to be impressed by the overall quality of ARL's technical staff and their work and applauds ARL for its clear, passionate concern for the end user of its technology--the soldier in the field--and for ARL's demonstrated mindfulness of the importance of transitioning technology to support immediate and longer-term Army needs. ARL staff also continue to expand their involvement with the wider scientific and engineering community. In general, ARL is working very well within an appropriate research and development (R&D) niche and has been demonstrating significant accomplishments.
 

 

An Assessment of the National Institute of Standards and Technology Information Technology Laboratory: Fiscal Year 2011 (LAB)

Released 09/29/11

Since 1959, the National Research Council (NRC), at the request of the National Institute of Standards and Technology (NIST), has annually assembled panels of experts to assess the quality and effectiveness of the NIST measurements and standards laboratories. In 2011, the NRC evaluated three of the six NIST laboratories: the Center for Nanoscale Science and Technology (CNST), the NIST Center for Neutron Research (NCNR) and the Information Technology Laboratory (ITL). Each of these was addressed individually by a separate panel of experts; this report assesses ITL.

Report in Brief (PDF)
 

 

An Assessment of the National Institute of Standards and Technology Center for Nanoscale Science and Technology: Fiscal Year 2011 (LAB)

Released 09/29/11

Since 1959, the National Research Council (NRC), at the request of the National Institute of Standards and Technology (NIST), has annually assembled panels of experts to assess the quality and effectiveness of the NIST measurements and standards laboratories. In 2011, the NRC evaluated three of the six NIST laboratories: the Center for Nanoscale Science and Technology (CNST), the NIST Center for Neutron Research (NCNR) and the Information Technology Laboratory (ITL). Each of these was addressed individually by a separate panel of experts; this report assesses CNST.
 

 

An Assessment of the National Institute of Standards and Technology Center for Neutron Research: Fiscal Year 2011 (LAB)

Released 09/29/11

Since 1959, the National Research Council (NRC), at the request of the National Institute of Standards and Technology (NIST), has annually assembled panels of experts to assess the quality and effectiveness of the NIST measurements and standards laboratories. In 2011, the NRC evaluated three of the six NIST laboratories: the Center for Nanoscale Science and Technology (CNST), the NIST Center for Neutron Research (NCNR) and the Information Technology Laboratory (ITL). Each of these was addressed individually by a separate panel of experts; this report assesses NCNR.

 

 

An Assessment of the National Institute of Standards and Technology Physics Laboratory: Fiscal Year 2010 (LAB)

Released 09.30.10

The mission of the NIST Physics Laboratory is to support U.S. industry, government, and the scientific community by providing measurement services and research for electronic, optical, and radiation technology. In this respect, the laboratory provides the foundation for the metrology of optical and ionizing radiations, time and frequency, and fundamental quantum processes, historically major areas of standards and technology.
The Panel on Physics visited the six divisions of the laboratory and reviewed a selected sample of their programs and projects.
 

 

An Assessment of the National Institute of Standards and Technology Materials Science and Engineering Laboratory: Fiscal Year 2010 (LAB)

Released 09.30.10

The Materials Science and Engineering Laboratory (MSEL) of the National Institute of Standards and Technology (NIST) works with industry, standards bodies, universities, and other government laboratories to improve the nation's measurements and standards infrastructure for materials. A panel of experts appointed by the National Research Council (NRC) assessed the four divisions of MSEL, by visiting these divisions and reviewing their activities
 

 

An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010 (LAB)

Released 09.30.10

The mission of the Manufacturing Engineering Laboratory (MEL) of the National Institute of Standards and Technology (NIST) is to promote innovation and the competitiveness of U.S. manufacturing through measurement science, measurement services, and critical technical contributions to standards. The MEL is organized in five divisions: Intelligent Systems, Manufacturing Metrology, Manufacturing Systems Integration, Precision Engineering, and Fabrication Technology. A panel of experts appointed by the National Research Council (NRC) assessed the first four divisions.
 

 

An Assessment of the National Institute of Standards and Technology Center for Neutron Research: Fiscal Year 2010 (LAB)

Released 09.30.10

The National Institute of Standards and Technology [NIST] Center for Neutron Research (NCNR) is a national user facility whose mission is to ensure the availability of neutron measurement capabilities in order to meet the needs of U.S. researchers from industry, academia, and government agencies. This mission is aligned with the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life.

As requested by the Deputy Director of NIST, this book assesses NCNR, based on the following criteria: (1) the technical merit of the current laboratory programs relative to current state-of-the-art programs worldwide; (2) the adequacy of the laboratory budget, facilities, equipment, and human resources, as they affect the quality of the laboratory technical programs; and (3) the degree to which the laboratory programs in measurement science and standards achieve their stated objectives and desired impact.
 

 

An Assessment of the National Institute of Standards and Technology Building and Fire Research Laboratory: Fiscal Year 2010 (LAB)

Released 09.30.10

A panel of experts appointed by the National Research Council assessed the scientific and technical work of the Building and Fire Research Laboratory (BFRL) of the National Institute of Standards and Technology (NIST). The scope of the assessment included the following criteria: (1) the technical merit of the current laboratory programs relative to the current state of the art worldwide; (2) the adequacy of the laboratory facilities, equipment, and human resources, as they affect the quality of the laboratory technical programs; and (3) the degree to which the laboratory programs in measurement science and standards achieve their stated objectives and desired impact. 

 

2010_capabilities_for_the_futureCapabilities for the Future: An Assessment of NASA Laboratories for Basic Research (LAB, ASEB)

Over the past 5 years or more, there has been a steady and significant decrease in NASA's laboratory capabilities, including equipment, maintenance, and facility upgrades. This adversely affects the support of NASA's scientists, who rely on these capabilities, as well as NASA's ability to make the basic scientific and technical contributions that others depend on for programs of national importance. The fundamental research community at NASA has been severely impacted by the budget reductions that are responsible for this decrease in laboratory capabilities, and as a result NASA's ability to support even NASA's future goals is in serious jeopardy.
 

 2009_2007_2008_assess_ARL

2007-2008 Assessment of the Army Research Laboratory

This volume is the latest in a series of biennial assessments of the scientific and technical quality of the Army Research Laboratory (ARL). The current report summarizes findings for the 2007-2008 period, during which 95 volunteer experts in fields of science and engineering participated in the following activities: visiting ARL annually, receiving formal presentations of technical work, examining facilities, engaging in technical discussions with ARL staff, and reviewing ARL technical materials.

The overall quality of ARL's technical staff and their work continues to be impressive, as well as the relevance of their work to Army needs. ARL continues to exhibit a clear, passionate concern for the end user of its technology--the soldier in the field. While two directorates have large program-support missions, there is considerable customer-support work across the directorates, which universally demonstrate mindfulness of the importance of transitioning technology to support immediate and near-term Army needs. ARL staff also continue to expand their involvement with the wider scientific and engineering community.

This involvement includes monitoring relevant developments elsewhere, engaging in significant collaborative work (including the Collaborative Technology Alliances), and sharing work through peer reviews. In general, ARL is working very well within an appropriate research and development niche and has been demonstrating significant accomplishments.
 

 

2008: An Assessment of the National Institute of Standards and Technology

 

 

 

2007: An Assessment of the National Institute of Standards and Technology

 

 

 

2005: An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Years 2004-2005

 

 

2003: An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories, Fiscal Year 2003

 

 

2002: An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories, Fiscal Year 2002

 

 

2001: An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories, Fiscal Year 2001

 

 

2000: An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories, Fiscal Year 2000
 1999: An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories, Fiscal Year 1999