Welcome to the REF section of the BU Research Blog!
The Research Excellence Framework (REF) has replaced the Research Assessment Exercise (RAE) as the process for assessing the volume and quality of research in UK HEIs. As with the RAE, the results of the REF will determine the annual quality-related research (QR) grant distributed from HEFCE to HEIs in England.
The last REF exercise was REF 2014. Institutions made their submissions in autumn 2013 and the results were published in December 2014. The next REF exercise is officially still referred to by HEFCE as the post-2014 REF, however, we are anticipating that the next assessment taking place will be REF 2021. If this is the case then submission would be in autumn 2020, meaning this would be a seven year assessment period.
The REF assesses research excellence through a process of expert review, informed by indicators where appropriate. It is based on HEIs submitting evidence of their research activity and outcomes to units of assessment (UOAs) to be assessed by expert panels.
What do we know about REF 2021?
HEFCE, on behalf of the four UK funding bodies, consulted with the sector in early 2017 as to the shape of the next REF. This consultation was informed by Lord Stern’s independent review of the REF, published in July 2016. The consultation closed on 17 March 2017. The funding bodies intend to announce their initial decisons on the next REF later in 2017. Further information about the consultation is available here: HEFCE consultation on REF2021 – what you need to know.
BU governance: Prof John Fletcher, Pro Vice Chancellor, Research and Innovation, has established a REF Committee (all UOA Leaders plus colleagues from Professional Services as appropriate) as a sub-committee of URKEC. The REF Committee met for the first time in September 2014 and is scheduled to meet at least three times per year.
Assessment elements: These are not yet confirmed, however, the consultation proposes these to be predominantly the same as REF 2014 (outputs, environment, impact), but with the addition of new categories (institutional environment and impact) (see the diagram below).
The role of metrics in research assessment: HEFCE conducted an independent review to ascertain the extent to which metrics could be used in the assessment and management of research. This was reported in The Metric Tide report that concluded that metrics are not yet robust enough to replace peer review. The consultation proposes that metrics are used to inform peer review in some UoAs (similar to REF 2014). Further information is available here: http://www.hefce.ac.uk/whatwedo/rsrch/howfundr/metrics/.
REF 2014 permitted the submission of research outputs in any form providing they embodied original research. It is likely this will be the same in REF 2021.
Open access: HEFCE have confirmed that all journal papers and conference proceedings published from April 2016 will have to have been made freely available in an institutional (such as BURO) and/or subject repository at the time of acceptance in order to be eligible for submission to the next REF. Read the policy here: http://www.hefce.ac.uk/whatwedo/rsrch/rinfrastruct/oa/policy/. You can read about how to add your outputs to BURO and ensure you are compliant with HEFCE policy here: Adding outputs to BRIAN.
Extra points available: HEFCE have confirmed that extra points in the environment section will be awarded to institutions that can demonstrate:
- that they have taken steps towards making other outputs types available open access on their repositories (such as book chapters)
- that outputs are presented in a form that allows re-use of the work, including via text-mining
Citations: Citation data was provided to some sub-panels in REF 2014 to inform the peer review process. Anecdotal evidence suggests reviewers are using these to judge borderline papers or to compare papers, with a high citation count tipping the score into the higher classification. It is proposed in the consultation that citations will be used in REF 2021 in a similar way to REF 2014 (i.e. used to inform peer review in some UoAs).
The assessment of the environment part of REF 2014 was measured by a narrative plus metrics (PGR completions and research income awarded). Assessment was based on the vitality and sustainability of the research environment (e.g. culture, staff development opportunities, career breaks, targeted recruitment, etc). The consultation proposes to retain the environment element but to split it between an institutional environment narrative (7.5% weighting) and a UoA-level environment narrative (7.5% weighting).
The impact element of REF 2014 was measured via impact case studies and a strategy narrative. For the purposes of the REF 2014, impact was defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. Impact included, but was not limited to, an effect on, change or benefit to:
- the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding
- of an audience, beneficiary, community, constituency, organisation or individuals
- in any geographic location whether locally, regionally, nationally or internationally.
Impact included the reduction or prevention of harm, risk, cost or other negative effects.
For the purposes of the impact element of the REF 2014:
- Impacts on research or the advancement of academic knowledge within the higher education sector (whether in the UK or internationally) were excluded.
- Impacts on students, teaching or other activities within the submitting HEI were excluded.
- Other impacts within the higher education sector, including on teaching or students, were included where they extend significantly beyond the submitting HEI.
The consultation on REF2021 invites feedback from the sector on whether the definition and scope of impact should be broadened.
Impact evaluation: HEFCE commissioned RAND Europe to undertake a comprehensive evaluation of the impact assessment element of REF 2014. Their remit was to:
- Look at the benefit and burden associated with impact submission preparation
- Recommend possible refinements of the process for future exercises
- Identify good practice across the sector
Further information and the final report can be found here: http://www.rand.org/randeurope/research/projects/hefce-ref2014-impact.html
Impact evidence: The consultation proposes that the impact assessment period (i.e. the period in which the claimed impact(s) occurred) runs from 1 August 2013 to 31 July 2020. It’s important that you keep the impact of your research in mind and get into the habit of collecting evidence and other corroborating information as impacts are realised, which will be of great help in developing case studies for the next REF.