Lewis-Burke Associates LLC – January 2012

On January 9, the National Science Foundation’s (NSF) National Science Board (NSB) published a report titled NSF Merit Review Criteria: Review and Revisions. After extensive review and stakeholder consultation, the NSB concluded that Intellectual Merit and Broader Impacts should be maintained as the two review criteria used to assess NSF proposals. The report provides clarification on the meaning of the two criteria and how they should be applied in the merit review process.

As previously reported by Lewis-Burke Associates, an NSB Task Force on Merit Review was established in February 2010 to examine the effectiveness of the two Merit Review Criteria. The Task Force sought extensive stakeholder engagement including: interviews with senior NSF staff; a public consultation resulting in over four thousand responses from reviewers and Principal Investigators; data from two hundred Committee of Visitor Reports; and data from over 100,000 research proposals submitted to NSF between 2006-09.

NSB concluded that while the two criteria remain appropriate, there is a need to provide greater clarity on each of the criteria and explain how they relate to the NSF core principles. The updated review criteria definitions are:

• Intellectual Merit: The Intellectual Merit criterion encompasses the potential to advance knowledge; and
• Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

NSB also established a set of overarching principles for merit review:

• Support highest quality projects.
• Aggregate of NSF projects to contribute to societal goal.
• Appropriate evaluation and assessment of projects.

In addition to the new principles and definitions for the two criteria, NSB has provided revised guidance on elements that should be considered in the review process for both criteria. This guidance will replace the individual guidance currently provided for each of the two criteria.

The report notes that the Broader Impacts criterion is required as an element of merit review under the America COMPETES Reauthorization Act. The COMPETES Reauthorization states that the overall goals of the broader impact criterion should be increased or improved to address issues including: economic competitiveness; global science, technology, engineering, and mathematics (STEM) workforce competitiveness; participation of women and underrepresented minorities; partnerships with industry; preK-12 STEM education; undergraduate STEM education; public scientific literacy; and national security. However, NSB recommended that NSF not include this list or other examples of broader impacts in the criteria to avoid being too prescriptive or restricting the creativity of NSF researchers.

NSF will be implementing the enhanced merit review criteria in subsequent months.

The full report is available at http://www.nsf.gov/nsb/publications/2011/meritreviewcriteria.pdf.

The current criteria can be found at http://nsf.gov/pubs/policydocs/pappguide/nsf11001/gpg_3.jsp#IIIA.