Developing a funder reviewer impact toolkit

LILI are leading a new project to develop impact support for NIHR funder review panels.

Project updates are available here.

The project

As part of NIHR’s ambition to “maximise the impact of our funding”, NIHR has commissioned the University of Lincoln to lead a project to explore how best to support funding panel members to interpret and judge impact at the application stage. Building on previous work to develop Impact Literacy tools for academics, this project explores how funding panels currently approach impact assessment, and what – if any – guidance, training or tools might support them.

Rationale

Project background

Across the research sector, there is a significant and growing requirement for academics to translate their insights into benefits – or impacts – for non-academics. The point at which funding decisions are made throws up particular challenges related to research impact, not least becuase impact is so difficult to judge. this project seeks to identify and address the particular challenges facing funders and they make their funding decisions.

Objectives

The main objectives for the project are to:

  • To understand how funding reviewers assess impact
  • To gather the expertise and experience of reviewers, which may otherwise be lost as panel membership changes over time
  • To explore the need for, and subsequently develop if needed, a new ‘tool’ or guide to help panel members review

Methods

Objectives

The project uses the six stage ‘Intervention Mapping’ (Eldridge et al, 2012) development process: (1) precise needs of the target population are explored through both primary and secondary research, which determine (2) the goals and objectives of a new tool. The (3) most suitable methods to address the needs are identified, and (4) assembled into an intervention or tool. Explicit consideration is given to how the tool can be (5) implemented and adopted into sustainable practice and (6) evaluated.

Impact

The knowledge generated by this project, and the associated ‘tool’ for reviewers will:

  • Improve current and/or future reviewer’s impact literacy
  • Strengthen the robustness of impact-relating decision making in funding
  • Improve NIHR organisational memory for impact assessment
  • Contribute in turn to improved research-led benefits for health and social care

Timelines

The project starting in January 2021 and is working to the following timelines:

  • Needs assessment: Jan to May 2021
  • Goals and objectives: April to May 2021
  • Methods, strategies and programme development: June 2021
  • Implementation, adoption and evaluation: July 2021

For updates on progress, click here.

Impact literacy

Project background in more detail

About NIHR

Project updates

Impact literacy

These difficulties inherent to impact underpin the work led by Dr Julie Bayley (Lincoln) and Dr David Phipps (York University, Toronto) on impact literacy. In short, if we are to contribute to genuine and meaningful change in society, we need to understand why it is needed, what change needs to happen (and the indicators we may need to measure), how this can be done and by who. Impact literacy is both individual-level, with people deepening their own understanding, and also institutional, with organisations ensuring there is appropriate support in place to facilitate these activities. Bayley and Phipps have been collaborating with Emerald Publishing in recent years to develop new tools for the sector, including an impact literacy workbook, designed to support academics to build stronger, realistic and achievable impact plans. More on impact literacy can be found here, and in the following materials:

About NIHR

The National Institute for Health Research (NIHR) is nation’s largest funder of health and care research, annually investing £1.3billion in the people, facilities, and technology that enable research to thrive. NIHR oversees multiple funding programmes, and has a core mission to NIHR’s mission is to improve the health and wealth of the nation through research. To successfully embed impact across research practice, impact needs to be designed into all areas of the system and across the research lifecycle. To date there has been considerable attention on impact assessment within the sector, and extensive guidance and expert insights into developing impact plans, but relatively far less attention on the process of judging impact at the application stage. Whilst there is clear good practice, there is also a risk that left unchecked such assessments could be unstandardised, non-transparent and with learning not carried through to successive panel members.

Background

Across the research sector, there is a significant and growing requirement for academics to translate their insights into benefits – or impacts – for non-academics.  Highly visible exercises such as the Research Excellence Framework have raised the profile of the impact agenda, but the need to make a meaningful contribution to society is a longstanding feature of research, from inception to well beyond project closure.

A critical juncture in the research pathway is the point at which funding decisions are made. Here, written accounts of the planned research and its potential impact are assessed across multiple criteria by panels of experts, and decisions made on which projects receive money. However impact can be particularly difficult to judge given it is:

  • Non-templatable; there is no one size fits all meaning different projects may seek very different effects via very different means
  • Not guaranteed; impact is by definition beyond the confines of a controllable research study
  • Temporally distant, often occurring beyond the lifetime of the project under the funder’s governance.
  • Notwithstanding experience and skill, often not the core specialism of each reviewer

Project Updates