Tagged / consultation

HEFCE consultation on REF2021 – what you need to know

ref-logoIt hardly feels like the dust has settled on the results from the last Research Excellence Framework, yet here we are again – almost half way through another research assessment cycle with HEFCE proposing potential changes to the next exercise. It feels like I’ve been transported back to 2009 when HEFCE were running bibliometrics and impact pilots to inform REF2014. I have to remind myself that it is in fact 2017 and things have moved on. We are now talking about REF2021 (how time flies!) and HEFCE have launched a consultation on how this exercise might work. Whilst citations and impact are still on the agenda for discussion, they are joined by other substantial changes that, if adopted, will transform the shape and potentially outcome of the next exercise.

The proposed changes have been informed by Lord Stern’s independent review of the REF which was published last July. This made 12 recommendations on how the next REF could be strengthened, whilst reducing the bureaucracy and cost of running the exercise (read Jane Forster’s overview of the recommendations – Stern review of the REF: what next?). These recommendations have now been reviewed by the four UK HE funding bodies and, prior to Christmas, HEFCE published proposals to amend the REF to incorporate the recommendations. HEFCE’s proposals are now open for consultation with the sector.

The document itself is fairly lengthy and dense, and over 40 questions are put forward for consideration. The key proposals are:

  • All research-active staff to be submitted
  • The decoupling of staff from outputs
  • Outputs will no longer be portable across institutions
  • All outputs must be available in open access form (with some exceptions)
  • Impact will have a broader definition
  • Institutional environment narratives and case studies to be submitted

There are some excellent summaries available online, many of which provide thoughts on what the proposals could mean in practice. The following summaries are particularly informative and worth reading: HEFCE launches consultation on REF2021, Soft Stern or Hard Stern and Implementing REF2021.

BU will be submitting an institutional response to the consultation before the deadline of 17th March. As these are significant proposals, all staff will have the opportunity to contribute to the response. As a number of the key questions would benefit from a discipline-level response, UoA teams (UoA leaders, impact champions and output champions) are calling meetings within Faculties to hold discussions related to their own UoA. These will take place over the next month or so. Contact a member of your UoA team for further information. Staff not based in Faculties are also invited to comment on the proposals and can do so via their Director/Head of Professional Service.

The are various REF-focused conferences and events taking place over the next few months, organised by HEFCE, ARMA, etc. Interesting news from these will be posted to the Research Blog. HEFCE are anticipating significant sector-wide engagement with the proposals and have committed to reading all feedback received. It is anticipated that, following the consultation, the initial decisions regarding the shape of REF2021 will be published in summer 2017.

Stern review of the REF – contributions welcome

ref-logoBIS have initiated a review of the Research Excellence Framework (REF), chaired by the President of the British Academy, Lord Nicholas Stern.

The review aims to ensure that future university research funding:

  • is allocated more efficiently;
  • offers greater rewards for excellent research; and,
  • reduces the administrative burden on institutions.

The call for evidence was published on 28 January and BU will be submitting an institutional response to the review. You are invited to contribute to this response.

The call for evidence, question set and timeline are available on the I-drive: I:\R&KEO\Public\Stern Review of the REF

If you wish to contribute, use the ‘questions’ document to frame your responses. The internal deadline for sending your feedback to Julie Northam is Wednesday 17 February. The draft response will be discussed by the REF Committee on 29 February, and the final draft shared with UET on 10 March.

When considering your responses please bear in mind Bournemouth University’s response to the green paper on HE which contained similar questions on the future shape of the REF. You may find it helpful to consider other responses to the green paper to understand the range in perspectives provided by institutions and groups. You can search for responses online. Suggested responses you may wish to read include:

I look forward to receiving your feedback and suggestions. Any queries, please let me know.

Julie

REF review to be led by Lord Stern of Brentford

Lord SternSummary from press release:

The HE green paper (Fulfilling our potential: teaching excellence, social mobility and student choice) and the outcome of the Comprehensive Spending Review, both released in November 2015, mentioned that there would be a review UK research funding and the REF as a mechanism for allocating funding. Further information about this review was released on 16 December 2015 in a press release from Jo Johnson, Universities and Science Minister (Government launches review to improve university research funding). The aim of the review is to reduce the burden and cost to universities and government that has become associated with preparing for, and submitting to, the REF, therefore ensuring the government gets the most return for its investment.

The review will be chaired by the President of the British Academy and former World Bank Chief Economist Lord Nicholas Stern. He will be assisted by a high-level steering group of academic experts, including the Vice-Chancellor of Aston University, Professor Julia King, and the Past President of the Academy of Medical Sciences, Professor Sir John Tooke.

Lord Stern commented: “Research assessment should not unwittingly introduce incentives for perverse behaviour, nor should it be overly burdensome. Excellence, properly defined, must remain the central basis for allocating support and funding for research. We will explore ways in which a simpler, lighter-touch, system for the REF might be developed.”

ref-logoThe review aims to:

1. Investigate different approaches to the evaluation of UK higher education research performance which can encourage and strengthen the emphasis on delivering excellent research and impact, while simplifying and reducing the administrative burden on the HE sector.

2. Draw on the evidence from the evaluation of REF2014 and consider other models of research performance assessment, which could provide robust means of informing future research funding allocations.
3. Provide options for future iterations of the REF focusing on a simpler, lighter-touch method of research assessment, that more effectively uses data and metrics while retaining the benefits of peer review. The review should ensure that a future process identifies and supports excellent research across the UK, including dynamic changes in research quality and emerging areas of research excellence, retains the frequency of approach of the current REF arrangements (at 5-6 year cycle) and secures the confidence of the HE/academic sector.

The full terms of reference for the review, including membership of the steering group, are available here: REF review terms of reference.

The outcome is due to be delivered in summer 2016.

 

Comments from the sector:

russell group logoUnsurprisingly many of the criticisms of the review have focused on the membership of the nine-member steering committee which includes seven academics and vice-chancellors from UK universities, all but one from members of the Russell Group. There are also no representatives from Wales, Northern Ireland or the funding councils and only one university in Scotland. Lord Stern defended the committee membership, saying he had looked for panel members from a “range of subjects” and for “people who are outstanding”. “Outstanding people are necessary to recognise excellence,” he said. “They are of the highest intellectual quality” and had experience in “running things”. The panel was not meant to be “a parliament of universities”, he said, but added that “we want to hear from everybody” about the future of the REF.

There have also been murmurings that the REF already offers a good return on investment, with costs estimated to only be 2.4% of the total funds its results will inform. If this is the case then is it even posible or desirable to reduce costs further?

The THE labelled Lord Stern as a ‘REF sceptic’, quoting a joint letter he wrote with Sir Paul Nurse in which he asked: “Have criteria of quality become too narrow and formulaic in some subjects? Are researchers feeling pressured to adopt short-term horizons and a narrow focus, and chasing publication rather than following their own judgements on which are the most fruitful avenues for research and most likely to yield major outcomes?” He also questioned whether the impact element of the REF was “insufficiently deep and broad”. The letter added: “is the REF incentivisation of universities to hire stars in the closing months, like an imminent transfer deadline in the Premier League, really a way to build a long-term scholarly department?”.

The suggestion of a more metrics-based approach is concerning, especially taking into account the recommendations in the Metric Tide report (July 2015) that concluded that metrics are not yet sufficiently robust to replace peer review or to be relied upon any more than they were in REF 2014.

Lord Stern defended the review, saying there was no “foregone conclusion” about the results, despite the HE green paper and CSR presuming it would lead to changes to the REF system. Dr Wendy Piatt, Director General of the Russell Group said that the Russell Group was supportive of the review but would be concerned if it resulted in any dilution of the REF’s rigour and international reputation.

 

Further reading:

THE – Lord Stern review: no ‘foregone conclusions’ about future of REF, 22 December 2015

THE – REF sceptic to lead review into research assessment, 16 December 2015

Russell Group – Stern review of research funding, 16 December 2015

Times – Prove research is useful or lose funds, universities are told, 16 December 2015

University Alliance – University Alliance responds to the announcement of Stern Review of university research funding, 16 December 2015

Should metrics be used more widely in the next REF?

Back in 2008, after the dust was settling from the REF 2008 submission, HEFCE initiated a series of exercises to investigate whether bibliometric indicators of research quality (such as citation counts) could be used as part of the assessment for REF 2014. BU was one of 22 institutions that took part in the bibliometrics pilot, the result of which was that HEFCE concluded that citation information was not sufficiently robust enough to be used formulaically or as a primary indicator of quality but that there might be scope for it to inform and enhance processes of expert review in some disciplines. The REF 2014 guidelines stated that citation data would be provided for outputs submitted to all sub-panels in Main Panel A and some sub-panels in Main Panel B.

In April 2014, the Minister for Universities and Science asked HEFCE to undertake a fresh review of the role of metrics in determining quality, impact and other key characteristics of research undertaken in the HE sector. The review is being chaired by Professor James Wilsdon, Professor of Science and Democracy at the Science Policy Institute, University of Sussex.

HEFCE have launched a sector-wide call for evidence about research metrics and BU will be making an institutional response. BU colleagues are therefore invited to send feedback to me so that it can be considered as part of BU’s response. Colleagues are also invited to send individual responses to HEFCE.

Thinking back to 2008-09, I remember research metrics being an emotive subject and many researchers, both at BU and  across the sector, were extremely skeptical of their use in research assessment. Although bibliometrics have moved on a long way since then I think that there will still be concern as to whether metrics are robust enough to be used formulaically, particularly in the arts, humanities and social sciences.

HEFCE have asked that responses focus on the following issues:

1. Identifying useful metrics for research assessment.

2. How should metrics be used in research assessment?

3. ‘Gaming’ and strategic use of metrics.

4. International perspective.

Further information about the call for evidence is available here: http://www.hefce.ac.uk/whatwedo/rsrch/howfundr/metrics/

It is anticipated that the outcome of the review will inform the framework for the next REF assessment so it is vitally important that HEFCE receive a high quality and quantity of feedback from all disciplines.

If you would like to contribute to the BU institutional response, please add your comments to this response form and email it to me (jnortham@bournemouth.ac.uk) by Friday 30th May.

Open access and the post-2014 REF – consultation open

HEFCE, on behalf of the four funding councils, has launched a formal consultation on open access in the post-2014 REF (likely to be REF 2020).  The deadline for responses is 30th October 2013 and BU will be submitting an institutional response that reflects the views of the majority of staff.

To facilitate the production of the institutional response the Deans of Schools have been invited to each submit a School-based response taking into account the views of academic colleagues.  It is of paramount importance that academic and research staff engage with this consultation because:

  1. The proposals would significantly change how academics record outputs and add full-text publications to BRIAN/BURO in future.
  2. Some of the proposals are discipline-based and/or have the potential to affect disciplines differently.

 

open access logo, Public Library of ScienceOverview of proposals

The funding bodies propose the following criteria for open access:

  • Outputs should be accessible through a UK higher education institution (HEI) repository, immediately upon either acceptance or publication, though the repository may provide access in a way that respects agreed embargo periods.
  • Outputs should be made available as the final peer-reviewed text, though not necessarily identical to the publisher’s edited and formatted version.
  • Outputs should be presented in a form allowing the reader to search for and re-use content (including by download and for text-mining), both manually and using automated tools, provided such re-use is subject to proper attribution under appropriate licensing.

We propose that outputs fulfilling the following definition must meet these criteria to be eligible for submission to the post-2014 REF:

  • The output is a journal article or conference proceeding.
  • The output is published after a two-year notice period (from 2016 onwards).
  • The output lists a UK HEI in the ‘address’ field.

We are seeking views on which of the following approaches to exceptions would be most appropriate:

  • universal compliance of outputs meeting the definition, with exceptions on a case-by-case basis
  • a specified percentage for compliance. 

The full consultation document can be accessed here: http://www.hefce.ac.uk/whatwedo/rsrch/rinfrastruct/openaccess/

The consultation refers to the RCUK Policy on Open Access which can be accessed here: http://www.rcuk.ac.uk/research/Pages/outputs.aspx

 

How to contribute

To contribute to the consultation please send your comments using this form (consultation template) to your School contact by 11th October as follows:

ApSci – Prof Jim Roach

BS – Prof Roger Palmer

DEC – Prof Jim Roach

HSC – Prof Gail Thomas

MS – Stephen Jukes

ST – Prof Adele Ladkin (on behalf of Prof Keith Wilkes)

 

Timeline

Early October      Launch consultation internally

11th October       All individual feedback to be sent to the Deans

18th October       Each Deans to send a School response to Julie Northam

23rd October       Julie Northam to send a draft institutional response to URKEC and Deans

28th October       All final feedback to be sent to Julie Northam

30th October       Deadline for submitting final institutional response to HEFCE

Early 2014            HEFCE to announce final plans regarding open access and REF 2020

Have your say on the EU

BIS have asked BU to participate in a consultation focussing around the EU’s competences.  This is an opportunity for us to provide evidence to an informed and objective analysis.

If you would like to contribute to this consultation then the questions to be addressed, and more information about the consultation, can be found here: Balance of Competences Research and Development

Please send your response to the questions to Jo Garrad by c.o.p. 19th July.  In order to help me combine the institutional response, please clearly identify which questions you are addressing.

Many thanks for your cooperation and vital input.

Have your say about funding areas in Horizon 2020

So you want a say in how Horizon 2020 will be but not sure how? Well, there are a number of consultations open for you to have your say – participate and start to shape EC funding for the future.

Opening up Education through New Technologies: The objective of the consultation is to explore the need for EU action to promote the use of Open Educational Resources (OER) and of Information and Communication Technologies (ICT) in education. The consultation is carried out in the context of a Commission Communication on “Rethinking Skills” planned for the end of 2012 which will announce a new EU initiative on exploiting the potential contribution of ICTs and OERs to education and skills development. A further Commission Communication on this is expected in mid-2013.

Marine Knowledge 2020: From Seabed Mapping to Ocean Forecasting: The European Commission is proposing to create a digital seabed map of European waters by 2020 by collecting all existing data into one coherent database accessible to everyone. The consultation closes on 15 December 2012.

Fuel Cells and Hydrogen JTI under Horizon 2020: The Fuel Cell and Hydrogen Joint Undertaking (FCH JU), a Public Private Partnership (PPP) set up between the European Commission and industry, was established in 2008. The research community also later joined the initiative. Its objective was to significantly accelerate the market introduction of the fuel cell and hydrogen technologies, in order to realise their potential as way to achieve a lower carbon energy system. The consultation aims at collecting the views of the wider public on the fuel cells and hydrogen sector, on research and innovation in this field in Europe, and in particular on the possible continuation of the FCH JU for the implementation of FCH research in Horizon 2020.

Commission seeks views on access to research infrastructure

If you get a bit cheesed off with the EC and want to have your say, then consultations are definately a great opportunity for you. There is one currently running on the follow-up to Framework 7’s “integrating activities” for research infrastructure which aims to widen access and enhance the use of research facilities that are already running in Europe and beyond. The consultation is gathering input until 22 October on possible specific topics for these activities under Horizon 2020 in 2014-20. Now’s the chance to have your say

EU Green Paper for FP8 released!

The Green Paper has now been released and proposes major changes to EU research and innovation funding in order to make participation easier, increase scientific and economic impact and provide better value for money. The changes, to be introduced in the next EU budget after 2013, will cover the current Framework Programme for research, the Competitiveness and Innovation Programme, and the European Institute of Innovation and Technology.  Read the full summary analysis to get a head start on what future EU funding may look like.

Results of Consultations on Post-2013 EU Education Programmes

Reports with results of the 2010 consultations on the future of the Lifelong Learning Programme (LLP), the Erasmus Mundus and the Youth Programme post-2013 are now available. The GHK Results of the Public Consultation and Overview of the Public Online Consultation findings will feed into the design of the next generation of education and youth programmes which are currently being prepared by the European Commission.  It is expected that the European Commission will present a proposal for the new education and youth programmes in the autumn of this year to the European Parliament and Council.

Consultation on Developing the Strategic Innovation Agenda for Innovation & Technology

A stakeholder consultation as part of the process to develop the Strategic Innovation Agenda (SIA) for the European Institute of Innovation and Technology (EIT) is being held.  This is an important consultation as it will be the main opportunity for the research and innovation community to shape the future of the EIT. It sets out a number of key questions covering the overall core objectives, potential themes for future Knowledge Innovation Communities and the criteria on which these should be selected. The consultation will close on 30 June 2011 and it will be possible to either answer the online questionnaire or submit separate position papers covering the main issues covered in the questions. Website on EIT consultation

ESRC Demand Management consultation

ESRC logoThe Economic and Social Research Council (ESRC) has today published plans for how it will manage the increasing demand for its research funding. Their aim is to have fewer, high-quality applications so the best social science is funded in the most effective way.

They will be introducing an initial programme of measures of improved self-regulation and a change to their existing peer review practices and submission policies.

These measures include:

  • the introduction of an invited-only resubmission policy as of June 2011
  • revised sifting mechanisms (greater use of outline applications and earlier sifting for standard grants)
  • issue more tightly specified calls on managed mode schemes which address the ESRC strategic priorities

After 12 months of these initial measures the ESRC will review their effectiveness, to establish whether further steps need to be taken to manage demand. In case further steps are required the ESRC welcome your views on the potential options, particularly in relation to the following questions:

  • Which main demand management options are worthy of further development and why?
  • How might those options be further developed and refined?
  • Which, if any of the main demand management options, would you not consider for further development and why?
  • Overall, which of the options offers the best opportunities to effectively manage demand whilst ensuring the flow of high-quality research applications? Are there any further options which are not included in this paper whcich should be considered by us as part of our demand management strategy?

The deadline for the submission of responses is 16 June 2011. These should be completed using the form on ‘SurveyMonkey’ at https://www.surveymonkey.com/s/demand_management.

We would encourage all academic staff researching in the social sciences to respond to the consultation.

To ensure your research proposal stands the best chance of success use BU’s internal peer review scheme – the Research Proposal Review Service.