Tagged / research assessment

REF 2020 update

In approximately 7 weeks we will know the outcome of the REF 2014 exercise. It is hard to believe that it is almost a year since we submitted to the exercise and that the results are round the corner. Whilst the expert panels have been assessing the submissions this past year, HEFCE have been working hard to design and shape the post-2014 REF, currently being referred to as REF 2020. They are currently midway through a review of the role of metrics in research assessment to ascertain the extent to which metrics could be used in the assessment and management of research. They have commissioned RAND Europe to undertake an assessment of the impact element of REF 2014, part of which will include recommendations to the assessment of impact in future REF exercises. They are currently consulting on whether an international REF exercise, rather than a national one, is the way forward. And, arguably the most important announcement to date, they have confirmed their open access policy for the next REF which stipulates that in order to be eligible for submission to the next REF from April 2016 all journal papers and conference proceedings have to be made freely available in an institutional and/or subject repository at the time of acceptance. Outputs not made freely available in a repository at the time of acceptance after April 2016 will be exempt from inclusion.

The Research Blog’s REF pages have recently been updated and you can read more of what we know about REF 2020 here: http://blogs.bournemouth.ac.uk/research/ref/

As soon as we know more we will post it on the Blog. Until then we wait with anticipation for the REF 2014 results!

HEFCE are looking for views on a potential international REF in future…

HEFCE has published a survey inviting views on an internationalised system of research assessment.

This survey forms part of a project exploring the benefits and challenges of expanding the UK’s research assessment system, the Research Excellence Framework (REF), on an international basis. At the broadest level, this means an extension of the assessment to incorporate submissions from universities overseas.

This follows an invitation earlier this year from the then Minister of State for Universities and Science, David Willetts, for HEFCE to provide an opinion on the feasibility of an international REF. The project belongs in a wider context of international interest in the exercise, on which HEFCE frequently provides information and advice to higher education policymakers and university senior management from overseas.

The THE ran a story about this in April 2014: HEFCE looks at overseas links for research excellence 

Responses are invited from any organisation or individual with an interest in higher education research or its assessment. The survey will be open until Wednesday 12 November 2014.

The survey only has four questions –

1. What do you think the key benefits would be of expanding the REF internationally?

2. What do you think the key challenges would be in expanding the REF internationally?

3. In view of the potential benefits and challenges overall, how supportive would you be of further work to explore the issues in more depth?

4. Have you got any further comments relating to internationalisation of REF?

To complete the survey visit: https://www.surveymonkey.com/r/refinternationalisation

Should metrics be used more widely in the next REF?

Back in 2008, after the dust was settling from the REF 2008 submission, HEFCE initiated a series of exercises to investigate whether bibliometric indicators of research quality (such as citation counts) could be used as part of the assessment for REF 2014. BU was one of 22 institutions that took part in the bibliometrics pilot, the result of which was that HEFCE concluded that citation information was not sufficiently robust enough to be used formulaically or as a primary indicator of quality but that there might be scope for it to inform and enhance processes of expert review in some disciplines. The REF 2014 guidelines stated that citation data would be provided for outputs submitted to all sub-panels in Main Panel A and some sub-panels in Main Panel B.

In April 2014, the Minister for Universities and Science asked HEFCE to undertake a fresh review of the role of metrics in determining quality, impact and other key characteristics of research undertaken in the HE sector. The review is being chaired by Professor James Wilsdon, Professor of Science and Democracy at the Science Policy Institute, University of Sussex.

HEFCE have launched a sector-wide call for evidence about research metrics and BU will be making an institutional response. BU colleagues are therefore invited to send feedback to me so that it can be considered as part of BU’s response. Colleagues are also invited to send individual responses to HEFCE.

Thinking back to 2008-09, I remember research metrics being an emotive subject and many researchers, both at BU and  across the sector, were extremely skeptical of their use in research assessment. Although bibliometrics have moved on a long way since then I think that there will still be concern as to whether metrics are robust enough to be used formulaically, particularly in the arts, humanities and social sciences.

HEFCE have asked that responses focus on the following issues:

1. Identifying useful metrics for research assessment.

2. How should metrics be used in research assessment?

3. ‘Gaming’ and strategic use of metrics.

4. International perspective.

Further information about the call for evidence is available here: http://www.hefce.ac.uk/whatwedo/rsrch/howfundr/metrics/

It is anticipated that the outcome of the review will inform the framework for the next REF assessment so it is vitally important that HEFCE receive a high quality and quantity of feedback from all disciplines.

If you would like to contribute to the BU institutional response, please add your comments to this response form and email it to me (jnortham@bournemouth.ac.uk) by Friday 30th May.

Seatbelts, sleepless nights, REF babies and a big yellow button: my reflections on REF 2014

Last week we pushed the Big Red Button (actually many big, yellow buttons) after many years of hard work preparing and finessing every last bit of the University’s submission to REF 2014.  I first got involved with REF in late 2008 which seems like a distant memory now.  HEFCE were consulting with the sector on bibliometrics and the role that citations should play in the REF which at that point still didn’t have a submission date.  BU was lucky enough to be one of 22 institutions taking part in the bibliometrics pilot to test the reliability and validity of citations and ways of identifying authors and papers in the large publication databases, Scopus and Web of Science.  There were a lot of meetings in London and so one of my first introductions to REF was standing on a cold, dark train platform at 6am with Anita Somner, waiting to get the train to one of the REF events!  In December 2008 the RAE 2008 results came out and that prompted a series of RAE/REF roadshows – in essence Prof Nick Petford, the then PVC, and I visiting all Schools to talk about the RAE results and introducing people to REF and how it was likely to be different.

Then a lot of the central REF drive died down which on reflection was a shame but also inevitable.  In 2009 all we knew about the next REF was that it would comprise outputs, environment and impact, but the key information about the assessment was still unknown.  For example, we didn’t yet know what the role of citations would be and in which UOAs this would apply, we didn’t know how the impact element would be assessed, the weightings weren’t agreed, we didn’t know what the environment template would look like or what information would be required.  With so much unknown it was very difficult to prepare anything other than for the outputs element and so the message given out was to concentrate on publications, getting them in the strongest outlet possible (we didn’t even know at this stage whether the number of outputs required per individual would be 3, 4 or 5!), and much of this work was driven from within the Schools.

Fast forward to 2010.  We had a change of senior leadership at BU with Prof John Vinney becoming VC and Prof Matthew Bennett taking the strategic lead for RKE (officially becoming PVC in January 2011).  There was still a lot undecided about REF; HEFCE had finished the bibliometrics pilot and were currently undertaking an impact pilot to test how best to assess this part of the REF.  At BU the new leadership provided by John Vinney and Matthew Bennett kickstarted our central REF preparations.  John established the REF Academic Steering Group in summer 2010 with a remit to take the strategic lead of BU’s preparations for REF.  UOA Leaders were identified in the then 12 subject areas in which we were considering submissions and they formed the REF Academic Leadership Team.  In the absence of templates or guidance from HEFCE we started work on the first drafts of the environment narratives and did the first trawl of outputs, inviting staff to submit up to four outputs for a light-touch review exercise that winter (282 individuals submitted a total of c. 1,128 outputs).  In autumn 2010 HEFCE published the results from the impact pilot, including some good practice examples of what we then knew to be one of the submission templates – the impact case studies.  Armed with this new information we undertook impact training with all of the Schools, using the infamous seatbelt example of how research can be undertaken and disseminated to achieve interim and then final impact and being informed by HSC that a reduction of the number of people killed in car crashes was actually a negative impact as there were less organs available for donation!  I have provided this lovely linear example of impact for posterity.  Around this time we also started to write up the first impact case studies, some of which evolved into the ones submitted last week.  In 2010 HEFCE confirmed what the UOAs would be and released the first list of who would be on the sub-panels.

2011 was a good REF year!  HEFCE confirmed in March how impact would be assessed in the REF and the official guidance document was finally released in June, providing us with something concrete on which to base our REF preparations.  It was the year we employed Sally Gates as the Research Communications Manager, focusing specifically on REF with the remit of working with colleagues to write the impact case studies in earnest.  We held the first of our HEFCE-supported REF events at BU attracting over 150 delegates from 39 institutions and speakers including the Deputy REF Manager Chris Taylor (HEFCE) and key academics involved in the impact pilot, including Prof James Goodwin (Age UK), Prof Peter Taylor-Gooby (University of Kent) and Prof Jim Griffiths (University of Plymouth).  This not only gave us an insight as to what was required for this still-very-unknown impact element, but also raised the profile of BU as a research university.  We held another mock exercise in winter 2011-12 to ask a selection of external reviewers to assess the draft environment narratives and impact statements.

I think it is fair to say that 2012 was a somewhat bizarre and full-on year, primarily due to the significant changeover in staffing but also because we were only a year from submission.  The year started with HEFCE releasing the Panel Working Methods and Criteria – key documentation detailing what the panels expected to see in the submissions.  This, along with the Guidance on Submissions published the previous year, became a lot of people’s bedtime reading for the next 23 months.  Despite swearing not to do so again (!) in February we held another HEFCE-supported REF event, this time focusing on how each of the Main Panels will assess research, and attracted over 150 delegates from 32 institutions, again putting BU on the research map.  We submitted the BU REF Code of Practice to HEFCE to the first of two possible opportunties and were pleased when it was approved first time (this wasn;t the case for a lot of other institutions).  Becca Edwards joined us in April as Public Engagement Officer and was immediately keen to be involved in the impact element of the REF, showing how public engagement could be a route to impact.  In the summer we held another mock exercise – this one focusing solely on outputs (265 individuals and c. 1,325 outputs).  Then bizarrely the three key REF staff in the R&KEO had babies between September and December 2012, resulting in a changeover of staff supporting the preparations – Pengpeng Hatch replacing Anita Somner, Becca Edwards replacing Sally Gates and Rita Dugan replacing me.  In December we responded to HEFCE’s Survey of Submission Intentions, a rough approximation of which UOAs we might submit to, how many staff, and the areas of the impact case studies.  This information was used to determine whether additional expertise was required on the Sub-Panels.

And then we came to 2013 – the year of submission.  Did we feel prepared?  Kind of, although there was still an awful lot to do; in fact I would go as far to say that REF probably dominated most waking minutes of those closely involved on an increasing basis as the year progressed.  The final mock exercise was held in spring 2013 (322 individuals and c. 1,610 outputs) and was a full exercise including assessment of outputs, environment narratives, impact statements and impact case studies.  This was a huge amount of data to pull together and it was essential that it was undertaken well as the results, along with those from the exercise the previous year, would be used to determine staff selection decisions.  I came back from maternity in the midst of the review meetings that followed the mock exercise, very much a baptism of fire back into the REF.  Wherever possible we tried to emulate how the panels might assess the work in the real thing, for example, with a panel of research users assessing the impact case studies.  After these meetings RASG met with UOA Leaders in a series of gruelling meetings in a very stuffy room in Christchurch House to go through each individual member of staff and their output scores, determining who’s outputs would be included in the final submission.  These decisions were ratified by the VC in July, decisions were then relayed to staff and an appeals period ran during the summer.  During this period we continued to get new outputs externally reviewed and considered, and had c. 100 new outputs assessed during this time.  The summer saw the return of Sally Gates from maternity leave and she took on the responsibility for rewriting and finalising the impact case studies, by all accounts doing a fantastic job.  In the autumn a huge amount of work went into finalising and finessing the environment narratives and impact statements, primarily involving the UOA Leaders, Profs Matthew Bennett and John Fletcher, Becca Edwards and myself.  I know from personal experience that a lot of sleep was lost during this time and there were many iterations of the narratives written.

Tuesday 26th November, 9:30am – button pushing time.  After checking, rechecking and checking the data again it was time to submit.  The #ref2014 hashtag on Twitter had gone crazy with institutions posting that they had made their submissions and now it was BU’s turn.  There were lots of submit buttons and they were all yellow – submit, validate, submit, declare and submit, submit, are you sure you want to submit?!  We pressed them all and that was it – BU’s REF 2014 submission was sent to HEFCE only to be acknolwedged with a rather bland, system generated email confirming receipt of our submission.  Sadly there were no fireworks or party poppers or massive thanks from HEFCE for all the hard work that went into preparing every last part of the submission.  REF has involved hundreds of people from across BU – from the UOA Leaders to those academics who have produced outputs, won grant funding or supervised doctoral students, from the REF Circs Board to the REF Appeals Panel, from the RKE Ops and Graduate School staff who support research activity and checked the REF data, to the members of RASG – the REF submission is the culmination of everyone’s hard work over the past few years and for that reason I am immensely proud to have been involved.  Roll on the results in December 2014!

And you may be surprise, amazed or even horrified to know that we have already started planning for REF2020…!

The Big Red Button

Actually it was yellow and there were several!  And yes we submitted our REF submission this morning; something of an anti-climax to be honest after three years of preparation and a huge amount of work by a large number of people especially over the last few weeks.  In terms of statistics we have:

  • submitted in eight units, notably for the first time in Psychology as well as in Leisure and Tourism;
  • 33% of eligible academic staff have been returned, up by 10% on RAE-2008 with a growth of 15% in eligible staff over the same period;
  • just over 40% of eligible staff were considered for selection;
  • our biggest submission, just short of 30 FTE, and is Geography/Archaeology;
  • we have submitted 22 Impact Case Studies and prepared many more. 

These numbers and statistics do not reflect the huge amount of work done by our UoA Leaders and their advisors, or the academics who have contributed the outputs to be returned and we salute you all for your work.  But in truth this is not the work of a few but a collective endeavour – academic and non-academic colleagues – a tribute to us all.  Without the selfless work of academics covering teaching while others have focused on research, without others generating RKE income, or supervising PGR students our collective success would not have been possible.  As such it is something that we should all feel proud of since we have all contributed whether returned with outputs or not.  As such we should be proud, whatever the outcome next December, of what we have achieved together.  The blood, sweat and toil is still written large in a few peoples’ mind at the moment; but boy will it be worth it and thank you!

BU REF2014 Open Forums next week

Next week there will be two BU REF2014 Open Forums to provide the opportunity for REF eligible staff to find out more about the provisional thresholds for the BU REF2014 staff selection process and to ask relevant questions.

Please find details of the events below:

Talbot Campus, 10 June 2013, 10am to 11am, Coyne Lecture Theatre, the Thomas Hardy Suite, Poole House

Lansdowne Campus, 13 June 2013, 9am to 10am, EB306, the Executive Business Centre

You can attend either one of the forums and there is no need to pre-register for these events.

Please feel free to get in touch with me (jnortham@bournemouth.ac.uk) or Peng Peng Ooi (pengpeng.ooi@bournemouth.ac.uk) if you wish to find out more.


Want to find out about how BU will shape submissions for the REF? Then come to one of our open forums!

Back in July we posted a copy of the BU REF 2014 Code of Practice on the Research Blog (BU REF Code of Practice gets the green light!). The Code of Practice sets out the process that the University will take in shaping submissions for REF 2014, including how outputs, staff and UOAs will be selected for submission and is available from the BU Staff Intranet: BU REF 2014 Code of Practice.

Peng Peng has recently emailed all academic staff with a copy of the Code of Practice, as well as the procedure for raising individual staff circumstances which may have affected an individual’s ability to undertake research to their full potential during the assessment period (such as being part-time, maternity leave, or being an early career researcher).

We are holding a couple of open forums for staff to engage with the Code of Practice and the procedure for raising individual staff circumstances, and these are open to all staff to attend. You can ask questions to a select panel from the internal REF management team, including Prof Matthew Bennett, Julie Northam, Peng Peng Ooi, James Palfreman-Kay, Judith Wilson and a number of the UOA Leaders.

This is an excellent opportunity to ask any questions you have regarding the REF!

Two open forums are planned, details are: 


Talbot Campus

Date: 27 September 2012

Time : 11.30am to 1.00pm

Venue: The Wallace Lecture Theatre, Weymouth House


Lansdowne Campus

Date: 11 October 2012

Time: 3.00pm to 4.30pm

Venue: EB306, Executive Business Centre


There is no need to book – simply turn up!

Latest BU REF Highlight Report now available

The latest BU REF Highlight Report (#12) is now available for BU staff to download. It covers the period from January to July 2012.

Features in this report include information about:

You can access your copy of the report from the following location on the I-drive (just copy and paste the following into Windows Explorer): I:\R&KEO\Public\RDU\REF\REF preparations\REF highlight reports

Updates from the BU REF Academic Leadership Team (RALT) meetings

The BU REF Academic Leadership Team (RALT) meets every month to discuss matters relating to BU’s preparations for its submission to the Research Excellence Framework (REF) in 2013. The group is chaired by Matthew Bennett (Pro Vice-Chancellor) and consists of the 12 Unit of Assessment (UOA) Leaders, who are academics from across the different Schools in the University, the REF Communications Manager (Sally Gates), the Head of the Research & Knowledge Exchange Office (Julie Northam) and representatives from the BU REF Academic Steering Group (RASG). The meetings are administered and minuted by the Research Development Officer for the REF (Anita Somner/Peng Peng Ooi).

A summary of the most recent meetings is below for your information. The next meeting is scheduled for September due to staff taking their summer holidays (hopefully there will be some sunshine soon!).

17 May 2012

  • Feedback from external reviewers was presented by two UOA Leaders following the previous mock exercise on impact and environment over the winter 2011/2012.
  • The process for the current outputs mock exercise was outlined and deadlines given for returning all information prior to it being sent to the external reviewers.
  • An update was given on the BU REF Code of Practice and on the first phase of equality and diversity training for those involved in coordinating the REF at BU, which had recently taken place.
  • The REF Communication Manager gave an update on progress with developing the BU impact case studies and on the provision of social media training for academics.

18 June 2012

  • Progress with the collation of data for the outputs mock was discussed. Most of the paperwork had been received on time, which was great news!
  • Preparations for the full mock in spring 2013 were discussed along with the kind of support UOA Leaders might need.
  • Forthcoming REF deadlines for the provision of certain information ahead of the 2013 submission deadline were highlighted.
  • The REF Communications Manager gave a report on the outcomes of a series of meetings held recently with UOA Leaders discussing the BU impact case studies.


BU REF Code of Practice gets the green light!

Since submitting the BU Research Excellence Framework (REF) Code of Practice to the REF Team at HEFCE in April (see previous blog post for details), we have been waiting for their feedback as to whether our Code meets the requirements and guidance on equality and diversity considerations for our REF preparations and submission. We finally had confimation from HEFCE on Monday that the REF Equality and Diversity Panel had reviewed all the Codes that had been submitted in April and that our Code did indeed meet the REF Team’s criteria and is now officially approved! This is great news and means that we can now make it available here on the BU Research Blog.

A PDF copy of the Code and the recently published BU REF Frequently Asked Questions (FAQs) will be circulated to academic staff in due course but you can access your copy here in the meantime. For the BU REF FAQs, which accompany the Code, a new tab has been added to the Blog (see REF tab above). Here you’ll find a series of questions and answers on a range of topics that should hopefully help with any initial burning issues you may have about the REF and our preparations here at BU. If you have a question that is not answered in the FAQs, please do leave a response to the relevant FAQ page on the Blog and we will follow this up for you. Alternatively you can contact me or Julie Northam in the Research Development Unit via email.

For more information about the REF at BU, see the previous REF posts on the Blog by clicking on the ‘ref’ tag. You can also access additional information from the REF website.

BU summer mock REF exercise gets underway

The fourth in a series of mock Research Excellence Framework (REF) exercises is currently underway at BU, with the paperwork now on its way to the external reviewers who have been specially recruited for their expertise in their subject area. The reviewers will spend the next month or so going through all the forms and submitted outputs and will provide feedback on each submission.

After reviewing the forms, the reviewers will also be involved in a meeting involving the BU Unit of Assessment (UOA) Leader and three or four nominated BU academics from each UOA to discuss the submissions and the feedback given. These meetings are due to take place during the early autumn. Once all the review meetings have taken place, the reviewers’ comments and the outcomes from the meetings will be compiled – this information will then be fed back to academics by the relevant UOA Leaders to help in the further development of the REF submissions.

The next (and final) REF mock exercise is due to take place during February to April 2013; it will be a full dry run and will be the last chance for you to receive external feedback on your outputs before the final REF submission deadline on 29th November 2013.

In the meantime, I’d like to thank all the BU academics who have been involved in the current mock exercise for taking part and for responding promptly to any queries. Thanks also to the UOA Leaders for collating the information and submitting it on time, particularly as the deadlines were quite tight. Your joint cooperation is much appreciated!

For all previous REF postings on the BU Research Blog, click here.

BU REF Code of Practice available now!

As I mentioned in my previous post about the forthcoming outputs mock REF exercise, we have recently submitted our final draft REF Code of Practice document to the REF Team for their approval. We produced this document to help inform staff about the policies and processes around submitting to the REF in November 2013, as well as to meet the four UK higher education funding bodies’ request that each institution making a submission to the REF must develop and apply such a code when selecting staff to include in their submission.

The BU REF Code of Practice is the culmination of months of drafting and approval, and sets out the approach that will be, and has already been, taken by BU in preparing its REF2014 submission. It therefore includes important information about the schedule of mock REF exercises that you are able to take part in (Chapter 3), and about declaring any circumstances that may have affected your ability to produce the four outputs required (Chapter 5), along with other equality and diversity aspects. Staff eligibility and selection are also covered.

Throughout the document there are references to the relevant paragraphs within the key publications published by the REF Team, which are: Assessment Framework and Guidance on Submissions and Panel Criteria and Working Methods.

Because this document is still subject to the approval of the REF Team’s Equality and Diversity Advisory Panel (EDAP), the Code is currently available to BU staff via the new Staff Intranet under ‘Policies, Forms and Procedures/Research’. We will make it fully available here on the Blog once we’ve received the EDAP feedback, so watch this space!

Limber up for the next BU mock REF exercise

Just as the Olympic athletes are getting set for the 2012 games, so are we gearing up to launch the summer 2012 mock REF exercise for BU academic staff, this time looking at research outputs. This follows on from our previous light-touch review exercise, which was open to all academic staff and took place over the winter of 2010/2011.

This latest exercise will be the fourth in a series of different preparation exercises that have been held or are due to take place at BU ahead of the REF2014 submission deadline on 29 November 2013. Just as the athletes will have taken part in warm-up events and qualifiers to test their fitness and shine before selectors, so the mock REF exercises are a useful ‘dry run’ to give you a feel for what’s going to be required for the real thing. Equally, you’ll get valuable feedback on your outputs from external reviewers and your UOA Leader, and it will enable us to test out our data collection processes.

As in the light touch review, the outputs mock will be open to all academic staff. The relevant form will be distributed via your UOA leader in due course so it would be a good time to start thinking about which outputs you’re likely to put forward. Although REF2014 requires a maximum of four outputs, we’re giving you the opportunity to get feedback from the reviewers on up to six outputs, so make the most of this chance to really shape your outputs submission.

Also under starter’s orders – the BU REF Code of Practice will shortly be circulated to all staff, which outlines the timetable of REF preparation exercises and includes a host of other useful information related to how BU is preparing for REF2014. The final draft of this document has just been submitted to the REF Team for formal approval so watch this space for more information about this.

If you need any more information about the REF, have a look at all the previous blog posts that we’ve included here, or visit the new-look REF website. Alternatively, you can contact myself or Julie Northam in the Research Development Unit, or leave a comment below.

REF Panel Criteria summaries by UOA

The final REF Panel Criteria and Working Methods documentation was released by the REF Team (based at HEFCE) at the end of January.

As a handy guide to the information the RDU has summarised the headline information for the UOA in the documents listed below:

Allied Health Professions, Dentistry, Nursing and Pharmacy – UOA 3 panel criteria summary

Psychology, Psychiatry and Neuroscience – UOA 4 panel criteria summary

Earth Systems and Environmental Sciences – UOA 7 panel criteria summary

Computer Science and Informatics – UOA 11 panel criteria summary

General Engineering – UOA 15 panel criteria summary

Geography, Environmental Studies and Archaeology – UOA 17 panel criteria summary

Business and Management Studies – UOA 19 panel criteria summary

Law – UOA 20 panel criteria summary

Social Work and Social Policy – UOA 22 panel criteria summary

Sports and Exerise Sciences, Leisure and Tourism – UOA 26 panel criteria summary

Art and Design – UOA 34 panel criteria summary

Communication, Cultural and Media Studies – UOA 36 panel criteria summary

Happy reading! If you have any questions about the REF then please do let me know.

The second brilliant external REF event at BU!

On Wednesday this week BU hosted a REF Team-supported event for universities in the south of England explaining the content of the recently released REF Panel Working Methods and Criteria documentation. This was the second REF event that has been hosted at BU in the past 12 months. The first event was held in May 2011 and you can read about it here: http://blogs.bournemouth.ac.uk/research/2011/05/25/the-excellent-hefce-ref-event-at-bu/

The event, attended by over 150 delegates from 32 institutions, outlined the similarities between how the four Main Panels will review submissions, as well as focusing on the differences between the panels particularly in how they will define and assess impact in the context of the REF.

Chris Taylor, Deputy REF Manager, spoke about the technical aspects of the REF, e.g. the timetable, element weightings, and institutional REF codes of practice, and then looked at each of the three elements of the REF in depth – impact, outputs and environment.

Prof Stephen Holgate, Chair of Main Panel A, then delivered a very interesting presentation on the similarities between the four Main Panels which have been vastly improved since the sector-wide consultation on the Panel Working Methods documentation last autumn. The Panels have put in a significant amount of work to ensuring their working methods will be as simple, transparent and similar as possible which is excellent news.

After break there were four concurrent sessions, each focusing on one of the Main Panels. Event attendees could choose to attend one session. The session were led by:

  • Main Panel A – Prof Stephen Holgate (Chair of Main Panel A)
  • Main Panel B – Prof Philip Nelson (Chair of the General Engineering sub-panel)
  • Main Panel C – Prof John Scott (Chair of the Sociology sub-panel)
  • Main Panel D – Prof Bruce Brown (Chair of Main Panel D)

After the concurrent sessions, all presenters took part in a Q&A session back in Kimmeridge House.

One of the key messages of the day was that the sub-panels will not make use of journal impact factors, journal ranking lists, or other journal scoring information to inform the review of outputs. Citation data will be provided by the REF Team to sub-panels:

  • Main Panel A: Sub-panels 1-6
  • Main Panel B: Sub-panels 7-11
  • Main Panel C: Sub-panel 18

Research collaboration (e.g. links with other institutions, business and industry, international collaboration, etc) was also highlighted at numerous points throughout the event as being of particular importance in the environment element of the assessment.

Regarding impact, Prof Holgate stressed that the assessment of impact was not necessarily linked to the size of the population affected but to the reach and significance of the impact – for example, a 4* impact case study could be for a drug that cured three people or 3 billion people.

Also interesting was the focus on 4* research being that which is transformative research and that this could be the synthesis of knowledge and the identification of a new way of doing things. A review paper could therefore be assessed as 4* if it meets this definition. Prof Holgate remarked: “we are in an era of transformation. We want game changing outputs to be submitted to the REF”.

The event was closed by Prof Matthew Bennett at 1pm after which point event attendees networked over lunch. Feedback from attendees so far has been very positive!

If you attended the session then we’d love to know what you thought! Let us know by adding a comment to this post.

The slides will be available shortly via the Blog.

REF draft code of practice – BU academic staff comments invited

Over the past 18-24 months we have been working on a code of practice to set out the approach that is being taken at BU in preparing our REF submission. The document has largely been authored by Anita Somner, Judith Wilson, Dr James Palfreman-Kay and myself with input from Prof Kate Galvin, and has been approved by the REF Academic Steering Group and REF Academic Leadership Team. It is a requirement for all institutions submitting to the REF to have a code of practice in place which has been approved by the REF Equality and Diversity Advisory Panel (EDAP), and the initial deadline for submitting draft codes to the EDAP is midday on 27 April 2012.

We are now inviting all academic staff to comment on this version of the BU REF code of practice. Comments received will help to refine the document in advance of the final version being sent by the VC to the EDAP at the end of April. A feedback form is available for completion by academic staff wishing to comment on the draft document.

The draft code of practice and the feedback form are available from the I-drive: I:\CRKT\Public\RDU\REF\COP feedback

Responses should be emailed to Anita Somner by 5pm on Friday 16 March 2012.

I am happy to discuss the document with colleagues and/or to meet as appropriate. If you have any queries, please do let me know.



Southern Universities REF event at BU – 22 February 2012 – book your place NOW!

REF logo
BU is hosting a 1/2 day REF event on Wednesday 22 February 2012. All staff are invited to attend.
The final panel working methods and criteria documents are due to be published in January 2012. This event will provide an update on the current developments with the REF and the confirmed REF panel documentation, focusing specifically on the assessment of impact within each of the four Main Panels.
Each of the REF Main Panels will be represented. If you have any questions about the REF, how research will be assessed and graded, or how impact will be assessed then you should attend this event! 😀
The event is open to BU staff and external delegates. There are already 130 delegates registered to attend, representing 32 different universities.

The event is free to attend but booking is essential.

For further information (including the programme) and to register, visit HERE.

Southern Universities REF event at BU – 22 February 2012 – book your place NOW!

BU will be hosting a half day Research Excellence Framework (REF) event, supported by the REF team, on 22 February 2012 to which all staff are invited to attend.

Book your place now by completing the online registration form 

This event follows hot on the heels of the first REF event held at BU on 19 May 2011, to which over 150 delegates from 39 institutions attended (see our previous blog post – The excellent REF event at BU!).

The sector-wide consultation on the proposed REF panel criteria closed earlier this month and the final documents are due to be published in January 2012. This event will provide an update on the current developments with the REF and the confirmed REF panel documentation, focusing specifically on the assessment of impact within each of the four Main Panels.

The event will be open to BU staff and external delegates and the provisional programme is shown below.

Time Activity
09:30 – 10:00 Coffee and registration
10:00 – 10:30  REF Team overview of the assessment framework
Chris Taylor, Deputy REF Project Manager, REF Team
10:30 – 11:00 Similarities between the four Main Panel working methods and criteria
Professor Stephen Holgate, Chair of Main Panel A
11:00 – 11:30 Morning break
11:30 – 12:15 Panel specifics, differences and impact assessment
There will be concurrent sessions, one for each of the four Main Panels. Attendees choose which one to attend.

  • Main Panel A:   Professor Stephen Holgate, Chair of Main Panel A
  • Main Panel B:   Professor Philip Nelson, Chair of sub-panel 15 (General Engineering)
  • Main Panel C:   Professor John Scott, Chair of sub-panel 23 (Sociology)
  • Main Panel D:   Professor Bruce Brown, Chair of Main Panel D
12:15 – 13:00 Panel Q&A session with all participants
13:00 – 14:00 Lunch, networking and close

The event is free to attend but booking is essential.

Book your place now by completing the online registration form