Category / Research assessment

The excellent HEFCE REF event at BU!

Research assessment

Developing and Assessing Impact for the REF

Last week BU hosted a HEFCE-supported event for universities in the south of England outlining recent changes in how the quality of research in higher education is assessed.

The event, attended by over 150 delegates from 39 institutions, outlined the new Research Excellence Framework (REF) which includes a new assessment element focusing on research impact.

As Chris Taylor, Deputy REF Manager for HEFCE, explained: “REF will provide accountability for public investment in research and demonstrate its benefits.” He continued:

“Impact is defined as any contribution the research makes outside of academia. It is the higher education sector’s opportunity to shout about what it contributes to society.”

Professor Peter Taylor-Gooby (University of Kent), Professor Roy Harrison (University of Birmingham), Professor James Goodwin (Age UK), Dr Kathryn Monk (Environment Agency Wales) and Dr Mari Williams (RCUK) presented their experiences of assessing impact case studies in the REF pilot exercise. Professor Jim Griffiths (University of Plymouth) presented his experience of identifying and submitting impact case studies to the pilot exercise in the hope that others would learn from his experience.

Prevalent themes emerging from the pilot included the importance of a demonstrable chain of evidence from impact claim through to outcome, high quality research underpinning the impact claim and fostering the crucial relationship between academic and user.

Professor James Goodwin explained how research can change society for people’s benefit, stessing the importance of “converting research into a message that will influence people’s thinking”. He gave the recent removal of the default retirement age as an example of how this can influence policy.

The event closed with a Q&A session with all speakers, giving delegates the chance to obtain further clarity on the REF that will undoubtedly change the future of higher education research.

Matthew Bennett (BU’s PVC for Research, Enterprise and Internationalisation) said: “There has been sector-wide concern about how impact will be defined, collated and assessed in the REF, and this event provided excellent advice and guidance for academic staff likely to be submitting to the REF and those leading the submissions.”

The deadline for submitting submissions is November 2013 and the assesment will be made in 2014.

We will be adding further posts to the Research Blog focusing on the good practice shared at the event (such as defining impact, what makes a strong impact case study, etc) over the next few weeks.

REF event 19 & 20 May 2011 – REGISTRATION IS OPEN!!

Research assessment

REF logoBU is hosting a two-day REF event on Thursday 19 and Friday 20 May 2011. All staff are invited to attend.

The event is of interest to BU academic staff and anyone who will be involved in the BU submission to the REF.

There will be three separate sessions:

Session 1
Thursday 19 May 9am-2pm
This session will be open to BU staff and external delegates.
There will be presentations from the REF team at HEFCE, REF impact pilot panel members, and a REF impact pilot institution (University of Plymouth).

Session 2
Thursday 19 May 2pm-5pm
This session is only open to BU staff.
This session will provide BU staff with the opportunity for internal networking, followed by a demonstration of BU’s new publications management system and a presentation on preparing a publication profile for the REF.

Session 3
Friday 20 May 9:45am-4:30pm
This session is only open to BU staff.
The focus of this session is the development of the BU impact case studies. There will be presentations of the impact case studies being developed at the moment.

All sessions will take place in Kimmeridge House and Poole House, Talbot Campus.

You must register separately for each session you will be attending.

See our previous REF Event blog post for further details. The provisional programmes are available on the registration forms (see links above).

REF event 19 & 20 May 2011 – SAVE THE DATE!

Research assessment

REF logoBU will be holding a two day Research Excellence Framework (REF) event on 19 and 20 May to which all staff are invited to attend.

Day 1 (open to BU staff and external delegates)
9am-2pm – this will be an external event supported by HEFCE to which all HEIs in the South of England will be invited. The focus will be on developing and assessing impact for the REF. There will be speakers from HEFCE, an academic from one of the impact pilot institutions (University of Plymouth), and some of the impact pilot panel members. The event is aimed primarily at academics likely to be submitted to the REF and UOA Leaders. It will provide a forum for networking and discussion around preparations for the impact element of the REF.

Day 1 (open to BU staff only)
2pm-5pm – There will be an opportunity for internal networking, a demonstration of the publications management system BU will soon be implementing, and a talk by Prof Matthew Bennett on building a publication profile for the REF.

Day 2 (open to BU staff only)
9am-4:30pm – the focus of Day 2 is the development of the BU impact case studies. The day will open with a presentation by Prof Matthew Bennett on what impact actually is, followed by presentations of the impact case studies being developed at the moment (3 per Unit of Assessment). These will run in 9 concurrent sessions with 4 presentations taking place in parallel during each session. The main aims of Day 2 are to get academics thinking about the impact case studies in a structured way, to identify resource requirements to maximise potential impacts, and the engage staff from M&C with the case studies being developed. In addition this is a great opportunity to showcase the excellent research that is undertaken at BU, to meet colleagues from other Schools, and to stimulate ideas for future research collaborations.

The event is free to attend but booking is essential. Booking will open next week – further details to follow!

REF Highlight Report #8

Research assessment

REF logoThe latest REF Highlight Report is now available from the Research Intranet.

Key points include updates on:

  • progress with the UOA Action Plans
  • the REF two-day event to be held at BU in May (19th/20th)
  • the second mock exercises for UOAs 7 and 26
  • the RASG and RALT meetings held in March

You can access the full document from here: REF Highlight Report #8

RCUK Research Outcomes Project

Research assessment

In an effort to record the outcomes and impact of research beyond the end of a funded project, Research Councils UK (RCUK) have been developing a system to gather the relevant quantitative and qualitative evidence. The information collated will be an important part of the Research Councils’ strategy development, and will be crucial in demonstrating the benefits of RCUK-funded research to society and the economy.

Currently, the Research Outcomes Project remains on course to go live in summer 2011, although a full roll out may not occur until autumn. Grant holders will be required to provide information on the following output types:

  • Publications
  • Collaboration
  • Communication
  • Exploitation
  • Recognition
  • Staff Development
  • Further Funding
  • Impact

The idea is for the system to allow outcomes to be reported at any point during the funding agreement and beyond, recognising that impacts from research are often realised some time after funding agreements have been completed.

The project team are engaging with users to ensure the system will be easy to use and aligned as far as possible with universities’ own research management information systems. A focus group of research managers provided feedback on the system, which is currently being built, and a pilot exercise is due to take place in August 2011.

BU is currently following the development of the Research Outcomes Project and will be looking at ways to help academics provide the required data in due course. In the meantime, please contact Anita Somner if you have any questions. To raise queries or concerns, or to make suggestions to RCUK, you can use the project email address: researchoutcomes@rcuk.ac.uk .

Journal Impact Factors Explained

Guidance, Research assessment

There is often some confusion around Journal Impact Factors in terms of where they come from, how they’re calculated and what they mean. Hopefully the following will provide a brief explanation.


What are Journal Impact Factors?
Journal Impact Factors are just one of a number of journal analytical measures that form part of an online resource provided by Thomson Reuters on their Web of Knowledge called Journal Citation Reports® (JCR), which covers journals in the sciences, technology and social sciences. JCR provides a facility for the evaluation and comparison of journals across fields within the subject areas covered.

Other publications databases may provide their own tools for bibliometric or citation analysis (such as Elsevier’s Scopus) but Journal Impact Factors are only found on the Web of Knowledge.

A Journal Impact Factor is the average number of times that articles from a particular journal published in the past two years have been cited in the JCR year.

How are Journal Impact Factors calculated?
Journal Impact Factors are calculated by dividing the number of citations to articles published by a particular journal in the JCR year by the total number of articles published in the two previous years. For example, an Impact Factor of 2.5 means that, on average, the articles published in that journal up to two years ago have been cited two and a half times. Citing articles may be from the same journal although most citing articles are from different journals.

The number of articles given for journals listed in JCR primarily include original research and review articles. Editorials, letters, news items and meeting abstracts are usually not included in article counts because they are not generally cited. Journals published in non-English languages or using non-Roman alphabets may be less accessible to researchers worldwide, which can influence their citation patterns.

How are Journal Impact Factors used?
Journal Impact Factors can help in understanding how many citations journals have received over a particular period – it is possible to see trends over time and across subject areas, and they may help when you’re deciding where to publish an academic paper. However, as with all statistics, Journal Impact Factors should be used with caution and should ideally be combined with other metrics depending on how they’re being applied.

Equally, a journal’s Impact Factor is not necessarily a direct indicator of the quality of an individual paper published in that journal. Some published articles never receive any citations, for various reasons, even if they appear in a high impact factored journal.

Journal Impact Factors and the REF
Some of the assessment panels will be provided with citation metrics as part of HEFCE’s Research Excellence Framework (REF) in some subject areas, which will help inform the panel members’ judgements. However, journal impact factors or equivalent journal ranking systems (e.g. the ABS list) will NOT be used at all within the assessment process.