Science and Technology in America's Reinvestment
Measuring the EffecT of Research on Innovation, Competitiveness and Science
Click here to go to the updated STAR Metrics web pages.
The continued interest in getting accurate information about the impact of science (http://www.sciencemag.org/cgi/content/summary/326/5954/783-a), combined with the success of the Star pilot program has resulted in an followon project entitled STAR METRICS. STAR METRICS is envisioned to be a multi-phase project that will evolve with time in response to the demands of stakeholders.
The STAR METRICS will create a reliable and consistent inter-agency mechanism to account for the number of scientists and support staff that are on research institution payrolls supported by federal funds. STAR METRICS will build on this information in future to allow for measurement of science impact on economic outcomes, such as job creation, on scientific outcomes, such as the generation and adoption of new science, often measured by citations and patents, as well as on social outcomes such as public health. Second, it builds on and expands existing research tools with minimal burden for government agencies and grantees.
At each stage, stakeholders will assess the effectiveness of the previous work in reaching goals and moving the project forward, and modify the approach as needed. Three sets of “products” are involved:
- Standardized measures of the impact of science investments on job creation and retention
- Systematized, standardized and validated ongoing measurement of long-term impact of science investments: economic, scientific, social.
- Enabling a community of researchers in science of science policy.
A critical element of STAR METRICS is to work with a group of volunteer PIs and universities to participate in the initial tests.
One component is to develop a set of report templates on outcomes that will be useful to universities with their stakeholders. An example of some ideas can be seen here – comments very welcome: REPORT FOR UNIVERSITIES.
Another component is to work with the PIs to develop report substance and templates on outcomes that will be useful for them in connecting with the scientific and investor community, reducing their reporting burden, and documenting the impact of their research An example of some ideas can be seen here – comments very welcome: REPORT FOR PIS
The third is for the agencies to develop report substance and templates that demonstrate the initial and cumulative impact of science investments on a broad range of outcomes An example of some ideascan be seen here – comments very welcome: REPORT FOR AGENCIES
The STAR METRICS team will post examples of these reports on this site and on the Science of Science Policy website (http://cs.scienceofsciencepolicy.net) for public comment as the project develops.
If you are interested in joining the project as a PI or as an institution, please contact Julia Lane at firstname.lastname@example.org.
(STAR METRICS stands for Science and Technology in America’s Reinvestment – Measuring the EffecT of Research on Innovation, Competitiveness and Science)
Proposed STAR Approach
Voeller Presentation from May FDP Meeting
Science of Science Policy Roadmap
The pilot is drawing on the existing administrative records/systems of academic institutions and other new and existing data on the economic, social and scientific outcomes of federal science investments. It is not expected that the pilot will provide a set of definitive answers in time for the necessary ARRA reporting requirement, but it should provide guidance about the feasibility and value of the approach in the longer term.
The Federal Demonstration Project is a key test community for a pilot. To minimize the impact on the overall membership, the FDP involvement will began with a few representative institutions selected to give the pilot team the best chance to uncover the nuances of how large and small, private and public, broad coverage and specialty institutions receive, manage, disburse and report on their use of Federal funding.
The goal of the pilot team is to maximize our understanding of what can be achieved using existing data and minimize any extra effort on the part of the recipients to achieve the pilot and full system objectives. Understanding the details of university grants management, financial and reporting systems, many of which are based on one of three widely sold products is critical as well.
- University of Texas at Austin
- University of Delaware
- George Mason University
- California Institute of Technology
- University of Alabama
- University of Massachusetts, Dartmouth