BU Research Blog

Last week we pushed the Big Red Button (actually many big, yellow buttons) after many years of hard work preparing and finessing every last bit of the University’s submission to REF 2014.  I first got involved with REF in late 2008 which seems like a distant memory now.  HEFCE were consulting with the sector on bibliometrics and the role that citations should play in the REF which at that point still didn’t have a submission date.  BU was lucky enough to be one of 22 institutions taking part in the bibliometrics pilot to test the reliability and validity of citations and ways of identifying authors and papers in the large publication databases, Scopus and Web of Science.  There were a lot of meetings in London and so one of my first introductions to REF was standing on a cold, dark train platform at 6am with Anita Somner, waiting to get the train to one of the REF events!  In December 2008 the RAE 2008 results came out and that prompted a series of RAE/REF roadshows – in essence Prof Nick Petford, the then PVC, and I visiting all Schools to talk about the RAE results and introducing people to REF and how it was likely to be different.

Then a lot of the central REF drive died down which on reflection was a shame but also inevitable.  In 2009 all we knew about the next REF was that it would comprise outputs, environment and impact, but the key information about the assessment was still unknown.  For example, we didn’t yet know what the role of citations would be and in which UOAs this would apply, we didn’t know how the impact element would be assessed, the weightings weren’t agreed, we didn’t know what the environment template would look like or what information would be required.  With so much unknown it was very difficult to prepare anything other than for the outputs element and so the message given out was to concentrate on publications, getting them in the strongest outlet possible (we didn’t even know at this stage whether the number of outputs required per individual would be 3, 4 or 5!), and much of this work was driven from within the Schools.

Fast forward to 2010.  We had a change of senior leadership at BU with Prof John Vinney becoming VC and Prof Matthew Bennett taking the strategic lead for RKE (officially becoming PVC in January 2011).  There was still a lot undecided about REF; HEFCE had finished the bibliometrics pilot and were currently undertaking an impact pilot to test how best to assess this part of the REF.  At BU the new leadership provided by John Vinney and Matthew Bennett kickstarted our central REF preparations.  John established the REF Academic Steering Group in summer 2010 with a remit to take the strategic lead of BU’s preparations for REF.  UOA Leaders were identified in the then 12 subject areas in which we were considering submissions and they formed the REF Academic Leadership Team.  In the absence of templates or guidance from HEFCE we started work on the first drafts of the environment narratives and did the first trawl of outputs, inviting staff to submit up to four outputs for a light-touch review exercise that winter (282 individuals submitted a total of c. 1,128 outputs).  In autumn 2010 HEFCE published the results from the impact pilot, including some good practice examples of what we then knew to be one of the submission templates – the impact case studies.  Armed with this new information we undertook impact training with all of the Schools, using the infamous seatbelt example of how research can be undertaken and disseminated to achieve interim and then final impact and being informed by HSC that a reduction of the number of people killed in car crashes was actually a negative impact as there were less organs available for donation!  I have provided this lovely linear example of impact for posterity.  Around this time we also started to write up the first impact case studies, some of which evolved into the ones submitted last week.  In 2010 HEFCE confirmed what the UOAs would be and released the first list of who would be on the sub-panels.

2011 was a good REF year!  HEFCE confirmed in March how impact would be assessed in the REF and the official guidance document was finally released in June, providing us with something concrete on which to base our REF preparations.  It was the year we employed Sally Gates as the Research Communications Manager, focusing specifically on REF with the remit of working with colleagues to write the impact case studies in earnest.  We held the first of our HEFCE-supported REF events at BU attracting over 150 delegates from 39 institutions and speakers including the Deputy REF Manager Chris Taylor (HEFCE) and key academics involved in the impact pilot, including Prof James Goodwin (Age UK), Prof Peter Taylor-Gooby (University of Kent) and Prof Jim Griffiths (University of Plymouth).  This not only gave us an insight as to what was required for this still-very-unknown impact element, but also raised the profile of BU as a research university.  We held another mock exercise in winter 2011-12 to ask a selection of external reviewers to assess the draft environment narratives and impact statements.

I think it is fair to say that 2012 was a somewhat bizarre and full-on year, primarily due to the significant changeover in staffing but also because we were only a year from submission.  The year started with HEFCE releasing the Panel Working Methods and Criteria – key documentation detailing what the panels expected to see in the submissions.  This, along with the Guidance on Submissions published the previous year, became a lot of people’s bedtime reading for the next 23 months.  Despite swearing not to do so again (!) in February we held another HEFCE-supported REF event, this time focusing on how each of the Main Panels will assess research, and attracted over 150 delegates from 32 institutions, again putting BU on the research map.  We submitted the BU REF Code of Practice to HEFCE to the first of two possible opportunties and were pleased when it was approved first time (this wasn;t the case for a lot of other institutions).  Becca Edwards joined us in April as Public Engagement Officer and was immediately keen to be involved in the impact element of the REF, showing how public engagement could be a route to impact.  In the summer we held another mock exercise – this one focusing solely on outputs (265 individuals and c. 1,325 outputs).  Then bizarrely the three key REF staff in the R&KEO had babies between September and December 2012, resulting in a changeover of staff supporting the preparations – Pengpeng Hatch replacing Anita Somner, Becca Edwards replacing Sally Gates and Rita Dugan replacing me.  In December we responded to HEFCE’s Survey of Submission Intentions, a rough approximation of which UOAs we might submit to, how many staff, and the areas of the impact case studies.  This information was used to determine whether additional expertise was required on the Sub-Panels.

And then we came to 2013 – the year of submission.  Did we feel prepared?  Kind of, although there was still an awful lot to do; in fact I would go as far to say that REF probably dominated most waking minutes of those closely involved on an increasing basis as the year progressed.  The final mock exercise was held in spring 2013 (322 individuals and c. 1,610 outputs) and was a full exercise including assessment of outputs, environment narratives, impact statements and impact case studies.  This was a huge amount of data to pull together and it was essential that it was undertaken well as the results, along with those from the exercise the previous year, would be used to determine staff selection decisions.  I came back from maternity in the midst of the review meetings that followed the mock exercise, very much a baptism of fire back into the REF.  Wherever possible we tried to emulate how the panels might assess the work in the real thing, for example, with a panel of research users assessing the impact case studies.  After these meetings RASG met with UOA Leaders in a series of gruelling meetings in a very stuffy room in Christchurch House to go through each individual member of staff and their output scores, determining who’s outputs would be included in the final submission.  These decisions were ratified by the VC in July, decisions were then relayed to staff and an appeals period ran during the summer.  During this period we continued to get new outputs externally reviewed and considered, and had c. 100 new outputs assessed during this time.  The summer saw the return of Sally Gates from maternity leave and she took on the responsibility for rewriting and finalising the impact case studies, by all accounts doing a fantastic job.  In the autumn a huge amount of work went into finalising and finessing the environment narratives and impact statements, primarily involving the UOA Leaders, Profs Matthew Bennett and John Fletcher, Becca Edwards and myself.  I know from personal experience that a lot of sleep was lost during this time and there were many iterations of the narratives written.

Tuesday 26th November, 9:30am – button pushing time.  After checking, rechecking and checking the data again it was time to submit.  The #ref2014 hashtag on Twitter had gone crazy with institutions posting that they had made their submissions and now it was BU’s turn.  There were lots of submit buttons and they were all yellow – submit, validate, submit, declare and submit, submit, are you sure you want to submit?!  We pressed them all and that was it – BU’s REF 2014 submission was sent to HEFCE only to be acknolwedged with a rather bland, system generated email confirming receipt of our submission.  Sadly there were no fireworks or party poppers or massive thanks from HEFCE for all the hard work that went into preparing every last part of the submission.  REF has involved hundreds of people from across BU – from the UOA Leaders to those academics who have produced outputs, won grant funding or supervised doctoral students, from the REF Circs Board to the REF Appeals Panel, from the RKE Ops and Graduate School staff who support research activity and checked the REF data, to the members of RASG – the REF submission is the culmination of everyone’s hard work over the past few years and for that reason I am immensely proud to have been involved.  Roll on the results in December 2014!

And you may be surprise, amazed or even horrified to know that we have already started planning for REF2020…!