Tagged / ref

REF 2021 – final decisions published

Research assessment

HEFCE kept their word and published the final decisions on REF 2021 in the autumn. Having issued the initial decisions on the Research Excellence Framework 2021 in September, the final decisions were published this week. HEFCE released its further decisions on staff and outputs on 21 November 2017.  These decisions have been informed by responses to key questions relating to staff and outputs and a survey of staff in scope for submission. This blog post provides a summary of the key decisions.

 

Submitting staff:

Previous REF/RAE exercises asked institutions to select staff for submission. The Stern Review in 2016 recognised how divisive this practice was and instead recommended that all research-active staff be returned to the next REF. HEFCE are implementing this recommendation by expecting all staff with a ‘significant responsibility for research’ to be submitted, provided they are ‘independent researchers’. What do these terms mean in practice? The HEFCE definition is:

“those for whom explicit time and resources are made available to engage actively in independent research, and that is an expectation of their job role. Research is defined as a process of investigation leading to new insights, effectively shared. Staff engaged exclusively in scholarship would not be considered to have a significant responsibility for research.”

Working with the REF 2021 main panels, HEFCE will provide further guidance on identifying staff with significant responsibility. This will be published in the guidance on submissions and panel criteria. This guidance will not prescribe a fixed set of criteria that all staff would be required to meet, but will set out a ‘menu’ of what HEFCE would consider may be appropriate indicators of significant responsibility.

 

Recognising that there are staff who have more significant responsibility for other activities, HEFCE will implement an approach whereby institutions, working with their staff and with guidelines, identify who is in scope for submission among staff meeting core eligibility criteria. HEFCE has defined the core eligibility criteria as:

‘Category A eligible’

  • academic staff with a contract of employment of ≥0.2 FTE
  • on the payroll of the submitting institution on the census date (31 July 2020)
  • whose primary employment function is to undertake either ‘research only’ (independent researchers only) or ‘teaching and research’
  • have a substantive connection with the submitting institution (i.e. BU)
  • for staff on ‘research only’ contracts, the eligible pool should only include those who are independent researchers, not research assistants

‘Category A submitted’ describes the staff from the ‘Category A eligible’ pool who have been identified as having significant responsibility for research on the census date.

Where the ‘Category A eligible’ staff definition accurately identifies all staff in the submitting unit with significant responsibility for research, the unit should submit 100% of staff. Where it does not accurately identify all staff in the submitting unit who have significant responsibility for research, institutions will need to implement processes to determine this and document this in a code of practice, approved by the relevant funding body with advice from the Equality and Diversity Advisory Panel (EDAP).

 

Submitting outputs:

  • The average number of outputs required per submitted FTE will be 2.5 (up from 2 outputs as previously suggested by HEFCE).
  • A minimum of one output will be required for each staff member employed on the census date (as expected).
  • A maximum of five outputs may be attributed to individual staff members (including those who have left) (down from 6 outputs as previously suggested by HEFCE).
  • Data on the distribution of outputs across staff in the unit, including staff who have left, will be provided to the sub-panels for consideration in relation to the assessment of the environment.

Output portability: A transitional approach is being adopted whereby outputs may be submitted by both the institution employing the staff member on the census date and the originating institution where the staff member was previously employed when the output was demonstrably generated. ‘Demonstrably generated’ will be determined by the date when the output was first made publicly available. This applies to the whole REF 2021 period.

Open access: The REF Open Access policy will be implemented as previous set out. This requires outputs within the scope of the policy (journal articles and some conference proceedings) to be deposited as soon after the point of acceptance as possible, and no later than three months after this date from 1 April 2018. Due to concerns around deposit on acceptance, a deposit exemption will be introduced from 1 April 2018 and remain in place for the rest of the REF 2021 publication period. This will allow outputs unable to meet this deposit timescale to remain compliant if they are deposited up to three months after the date of publication.

 

Number of impact case studies required

Submissions will include a total of one case study, plus one further case study per up to 15 FTE staff submitted, for the first 105 FTE staff returned (with a reduced requirement above this of one additional case study per 50 FTE staff). Submissions will therefore include a minimum of two case studies.

 

For the latest information, see our REF 2021 webpage.

REF 2021 workshops – what makes a 2*, 3* or 4* output?

BU research, Research assessment, RKE development framework

We have a series of externally-facilitated REF outputs workshops scheduled to take place in early 2018 as part of the RKE Development Framework. Each session is led by REF 2014 sub-panel member who will explain how the panel interpreted and applied the REF 2014 guidance when assessing the quality of outputs. The workshops are open to all academic staff to attend.

The expected learning outcomes from the workshops are for attendees to:

  • Gain insight into how the REF panels applied the REF criteria when considering the significance, rigour and originality of outputs;
  • Understand the differences between outputs scored 4*, 3*, 2*, 1* and Unclassified;
  • Gain insight into what is meant by ‘world leading’ and ‘internationally excellent’;
  • Understand how scores borderline cases were agreed and what the tipping points were to either break the ceiling into the higher star level or to hold an output back a star level;
  • Understand how panels used other information such as metrics, markers of journal quality or prior knowledge in output assessment;
  • Gain insight into how future outputs could be strengthened for REF2021.

 

We’ve got dates for half of the UOAs so far:

  • UOA 2/3 – Prof Dame Jill Macleod Clark – date tbc (likely to be mid to late February 2018)
  • UOA 4 – Prof Marion Hetherington – 10 January 2018
  • UOA 11 – Prof Iain Stewart – 29 January 2018
  • UOA 12 – Prof Chris Chatwin – 8 January 2018
  • UOA 14 – Prof Jon Sadler – date tbc
  • UOA 15 – Prof Graeme Barker – date tbc
  • UOA 17 – Prof Terry Williams – 17 January 2018
  • UOA 18 – tbc
  • UOA 20/21 – Prof Imogen Taylor – 15 January 2018
  • UOA 23 – Prof Jane Seale – 26 January 2018
  • UOA 24 – tbc
  • UOA 27 – Prof Pat Waugh – 16 January 2018
  • UOA 32 – Prof Stephen Partridge – date tbc
  • UOA 36 – Prof Peter Lunt – date tbc

Bookings for these can be made via the Staff Intranet: https://staffintranet.bournemouth.ac.uk/workingatbu/staffdevelopmentandengagement/fusiondevelopment/fusionprogrammesandevents/rkedevelopmentframework/researchexcellenceframework/

REF2021 – initial decisions finally published

BU research, Research assessment

On Friday there was an exciting update from the REF Team based at HEFCE – they published the initial decisions on REF 2021. Whilst this does not include decisions regarding submitting staff, output portability or the eligibility of institutions to participate in the REF, it does include key decisions regarding the UOA structure, institution-level assessment, and the assessment weightings.

The decisions published on Friday are summarised below:

 

OVERALL:

Assessment weightings:

  • Outputs 60% (down from 65%)
  • Impact 25% (up from 20%)
  • Environment 15% (same but now includes impact strategy)

The move of the impact template from the impact assessment to the environment assessment means impact will actually contribute to more than 25% of the weighting (see impact section).

Assessment will continue to use the five-point REF 2014 scale (1*-4* and Unclassified).

UOA structure:

  • Total UOAs reduced from 36 to 34
  • Engineering will be a single UOA – UOA 12
  • REF 2014 UOA 17 will be restructured to form UOA 14: Geography and Environmental Studies and UOA 15: Archaeology
  • ‘Film and Screen Studies’ will be located and included in the name of UOA 33: Music, Drama, Dance, Performing Arts, Film and Screen Studies
  • HEFCE will continue consulting with the subject communities for forensic science and criminology to consider concerns raised about visibility. A decision is expected this autumn.

HESA cost centres will not be used to allocate staff to UOAs. Responsibility for mapping staff into UOAs will therefore remain with institutions.

 

TIMETABLE:

Impact:

  • Underpinning research must have been produced between 1 Jan 2000 – 31 Dec 2020.
  • Impacts must have occurred between 1 Aug 2013 – 31 Jul 2020.

Environment:

  • Environment data (such as income and doctoral completions) will be considered for the period 1 Aug 2013 – 31 Jul 2020.

Outputs:

  • The assessment period for the publication of outputs will be 1 Jan 2014 – 31 Dec 2020.

The draft REF 2021 guidance will be published in summer/autumn 2018 and the final guidance will be published in winter 2018-19. The submission will be in autumn 2020.

 

OUTPUTS:

Interdisciplinary research:

  • Each sub-panel will have at least one appointed member to oversee and participate in the assessment of interdisciplinary research submitted in that UOA.
  • There will be an interdisciplinary research identifier for outputs in the REF submission system (not mandatory).
  • There will be a discrete section in the environment template for the unit’s structures in support of interdisciplinary research.

Outputs due for publication after the submission date:

A reserve output may be submitted in these cases.

Assessment metrics:

Quantitative metrics may be used to inform output assessment. This will be determined by the sub-panels. Data will be provided by HEFCE.

 

IMPACT:

  • Impact will have a greater weighting in REF 2021 (25% overall plus impact included in the environment template and therefore weighting).
  • Harmonised definitions of academic and wider impact will be developed between HEFCE and the UK Research Councils.
  • Academic impacts will be assessed as part of the ‘significance’ assessment of the outputs and therefore not in the impact assessment.
  • Further guidance will be provided on the criteria for reach and significance and impacts arising from public engagement.
  • The guidance on submitting impacts on teaching will be widened to include impacts within, and beyond, the submitting institution.
  • Impacts remain eligible for submission by the institution in which the associated research was conducted. They must be underpinned by excellent research (at least REF 2*).
  • Impact case study template will have mandatory fields for recording standardised information, such as research funder, etc.
  • The number of case studies required – still not confirmed – HEFCE are exploring this in relation to the rules on staff submission and the number of outputs.
  • Case studies submitted to REF 2014 can be resubmitted to REF 2021, providing they meet the REF 2021 eligibility requirements.
  • The relationship between the underpinning research and impact will be broadened from individual outputs to include a wider body of work or research activity.

 Institutional-level assessment (impact case studies):

  • HEFCE will pilot this assessment in 2018 but it will not be included in REF 2021.

 

ENVIRONMENT:

The UOA-level environment template will be more structured, including the use of more quantitative data to evidence narrative content:

  • It will include explicit sections on the unit’s approach to:
    • supporting collaboration with organisations beyond HE
    • enabling impact – akin to the impact template in REF 2014
    • supporting equality and diversity
    • structures to support interdisciplinary research
    • open research, including the unit’s open access strategy and where this goes beyond the REF open access policy requirements

Institutional-level assessment (environment):

  • Institution-level information will be included in the UOA-level environment template, assessed by the relevant sub-panel.
  • HEFCE will pilot the standalone assessment of institution-level environment information as part of REF 2021, but this will not form part of the REF 2021 assessment. The outcomes will inform post-REF 2021 assessment exercises.

 

PANEL RECRUITMENT:

  • The sub-panel chair application process is now open (details available via the link).
  • The document sets out the plan for the recruitment of panel members (a multi-stage approach)

 

OUTSTANDING DECISIONS:

The announcement does not include decisions regarding submitting staff, output portability or the eligibility of institutions to participate in the REF. There is ongoing dialogue between HEFCE (on behalf of the funding councils) and the sector regarding this. The letter (accessed via the link above) sets out HEFCE’s current thoughts on these points and invites the sector to feedback by 29 September 2017.  BU will be providing feedback so if you have a view on this then please email me (jnortham@bournemouth.ac.uk).

 

SUMMARIES AVAILABLE:

I’m an academic at BU. Will I be submitted to REF 2021?

Research assessment

Good question and, although no firm decisions have yet been announced by HEFCE, it is looking increasingly likely that all academic staff at BU will be included in the REF 2021 submission, each with at least one output published between 2014-2020.

In the midst of the sector waiting with baited breath for the initial decisions from the UK funding bodies on this, and other REF questions, HEFCE held a webinar in July. During this webinar they shared some possible decisions with the sector (the webinar and the slides are available here on the HEFCE website). The key suggestions were:

  • 100% of academics with a “significant responsibility” to undertake research are likely to be included. It is unclear at this stage what “significant responsibility” means in practice, although it is anticipated this will be based on there being an expectation for an academic member of staff to undertake research.
  • Staff without a significant responsibility for research may be exempt from inclusion but auditable documentation would be required. This would need to explicitly evidence there is not an expectation of the individual to undertake research (examples given were workload models or career frameworks linked to the individual).
  • Everyone submitted is likely to need a minimum of 1 output. The average and maximum outputs per FTE are to be determined – in the consultation it was proposed these were an average of 2 outputs per submitted FTE and a maximum of six outputs per person.
  • There is likely to be a hybrid model for output portability (i.e. which HEI can submit the outputs authored by a member of academic staff who moves from one institution to another during the REF period) – HEFCE proposed two options:
    • Simple model whereby both old and new institutions can submit the outputs produced by the academic member of staff when he/she was employed at the old institution (this would, some might say unfortunately, result in double counting of outputs but this can probably be tolerated as it happens already in some cases, for example, where co-authors at different HEIs submit the same output).
    • Complex model whereby a census date and employment range date are used to determine which outputs can be submitted by which institution.

Whilst these are not yet firm decisions (these are expected in two communications – one on staff and outputs in the autumn and one on everything else later this month), these are the clearest indications yet that all academic staff at BU will be included in REF 2021, each with at least one output.

For further information on REF 2021, see BU’s REF 2021 webpage. If you have any queries, please contact Julie Northam or Shelly Anne Stringer.

REF Main Panel Chairs announced

Research assessment

The main panels will provide leadership and guidance to the sub-panels that undertake the REF assessment. As chairs designate, the appointees will at first advise the funding bodies on the initial decisions and on the further development of the framework. They will take up their roles as chairs later in the year*, once the outcomes of the ‘consultation on the second REF’ are announced and further appointments to the REF panels have been made.

The Main Panel Chairs (designate) for each of the four main panel areas are:

Biographies for the Main Panel Chairs are available here: Biographies

*Interesting to note that HEFCE have reaffirmed their previous commitment to announce the outcomes of the consultation later this year, despite rumours this would either be delayed or result in a second technical consultation.

REF 2021 – stocktake exercises

Research assessment

With the publication of the Stern Review last summer and the funding bodies’ Second Consultation on the REF earlier this year, there’s been a lot of discussion at BU and across the sector around REF 2021 lately. Despite this, and indeed because of this, we’re still none the wiser as to what the next REF will look like. Like many other universities, we are progressing with our internal preparations whilst we await the publication of the initial decisions from the funding bodies’ in response to the feedback to their consultation (predicted to be later this year).

One of the ways BU is preparing is by running a stocktake exercise to see what outputs academic staff have published since 1 January 2014 and what potential impact BU research is having. Not only will this provide a summary of progress c. half way through the REF assessment period, it will also enable resources to be allocated to support further high-quality outputs and to accelerate research impact.

The stocktake exercise is being run in two cohorts:

  • Cohort 1 takes place this summer and involves UOAs – 2, 3, 4, 12, 22/23, 25, 34 and 36.
  • Cohort 2 takes place this autumn and involves UOAs – 11, 17 (archaeology), 17 (geography and environmental studies), 19, 20, 26 and 29.

The process will be the same for each cohort. On the outputs side, we are changing from individuals self-nominating for their inclusion in the exercise to a model where all academic staff (with a research-only or a teaching and research contract) are automatically included. This ensures the exercise is fully inclusive whilst reducing the burden on individual academic staff. In terms of impact, we are changing from colleagues writing impact case studies to inviting them to attend a meeting and deliver a short informal presentation of their research, its impact and their plans for generating further impact, followed by a discussion with the panel. This is linked to the launch of the new impact tracker in BRIAN.

The stocktake exercises are designed to be fully inclusive, positive and developmental. Further information about the REF is available on the Research Blog’s REF webpage.

FHSS – New Impact Champion

Research assessment

I just wanted to take this opportunity to introduce myself as the new Impact Champion for the Faculty of Health and Social Sciences. When I began my research journey several years ago I knew I wanted my research to make a difference. It is this on-going desire to make a difference which led me to pursue this role of Impact Champion.

Part of my role is to take the lead in supporting academics to develop, maintain and ensure that research impact aligns with our core principle of ‘helping to make people’s lives better’. I will also be involved in developing impact case studies, and proactively accelerating impact in relation to REF.

Please don’t hesitate to get in touch if there is anything I can do to support you in your research impact.

Clare Killingback: ckillingback@bournemouth.ac.uk

HE policy update w/e17th March 2017

EU, policy, Research assessment, Research news, Research Training, RKE development framework

Brexit:

  • Research Professional illustrates the Brexit threats to research positioning and job losses by highlighting the difficulties facing an EU astronomy consortium. The consortium represents seven countries, led by the UK, but will move headquarters to an EU member state from January 2021. The move means the UK will lose the project’s leadership and the 12 UK universities may not continue post-Brexit. Research Professional notes that while access to research infrastructures is available to non-EU states, the EU membership plays a significant role in decisions on where to locate facilities. Gerry Gilmore (the consortium leader, from University of Cambridge) stated:
    “The UK will lose substantial scientific leadership and influence in the EU. There is going to be bad news all around. I don’t think people realise how many new jobs and new opportunities have just been destroyed.”
  • The EU (Notification of Withdrawal) Bill has survived the parliamentary process and received Royal Assent on 16th March (BBC). This bill allows the Prime Minister to notify the EU of the UK’s intention to withdraw from the EU. The Lords made two amendments to the Bill – one relating to Parliament having a “meaningful vote” on the final arrangements and one requiring a guarantee for EU citizens to remain in the UK. The bill was approved by the House of Commons, which rejected the Lords bill and then went back to the Lords under what is called “ping-pong”. The Lords voted again on both issues but the House of Lords majority backed down and the bill was passed. The PM is expected to trigger article 50 later in March.
  • 2018/19 EU student and staff guarantees: During oral questions in the Lords Baroness Royall of Blaisdon pressed the government spokesperson (Viscount Younger of Leckie) when announcements would be made regarding fees and access to loans for 2018/19 EU student starters. Leckie gave a side stepping response: “The noble Baroness makes the important point that there are uncertainties arising from Brexit, but the Government have moved rapidly to give assurances to this sector… “We have also provided similar assurances that EU nationals starting courses in 2016-17 and 2017-18 remain eligible for Research Council postgraduate support. As I have said, we will ensure that students starting in 2018-19 have the information well in advance”

International students:

  • The debate over the inclusion of international students in the long-term migrant numbers continues. Even senior ministers are rebelling – Boris Johnson, Phillip Hammond and Liam Fox have all protested, although Jo Johnson continues to toe the party line backing the PM’s stance to include international students within the original immigration statistics. Liam Fox spoke out this week about the value of overseas campuses.
  • On Monday the House of Lords defeated the government on the Higher Education and Research Bill (HERB), approving an amendment to prevent international students being counted as long-term migrants. The government have responded that “the proposed amendment would create a situation where we were potentially unable to apply basic visa checks, or impose conditions on a student visa. It would also mean that fresh primary legislation were needed just to make minor, technical changes to immigration rules.” (Wonkhe)
  • HERB is scheduled to have its third reading in the Lords on 22 March 2017 and then will go back to the Commons. The PM’s stance on international students seems rock solid (Financial Times) and Theresa May is not expected to waiver – the parliamentary ping pong regarding international students will surely make headlines over the coming weeks.
  • Meanwhile there are worries about student recruitment. Politics Home quotes an Office for National Statistics release stating the number of students coming to the UK dropped by 41,000 in 2016.

Higher Education and Research Bill:

  • The HE and Research Bill has finished its third reading in the House of Lords (although it will have to go back if the House of Commons makes any changes, as seems likely).  The report stage in the Lords is on 22nd March – usually only technical or minor amendments are made at this stage.  The current version of the bill as amended by the Lords is here.
  • The surprise amendment on international students is referred to above.
  • The government won the final vote on the proposed amendment that would have required UKRI and OfS to jointly revoke research degree awarding powers, the amendment was defeated. Wonkhe report that Lord Mackay made an impassioned speech noting that it was “extraordinary” that the OfS was not required to have any expertise or experience regarding research, and yet had the unilateral power to revoke research degree awarding powers, but to no avail. The Bill continues to say that research degree-awarding powers should be made by the OfS with advice from UKRI.

With long debates, late nights and a large number of amendments, it is fair to say that HERB has received an excellent level of scrutiny within the Lords. Lord Prior of Brampton notes: “Everyone who has contributed [to the Bill debates] can take some credit for having improved it considerably. For me, it is a good example of the value this House can bring to a Bill of this kind.”

HEFCE 2017/18 funding to universities: The grant letter details the overall funding to the sector for 2017/18. It includes doubled funding for the National Collaborative Outreach Programme (£60m pa), an additional £17m increase for mainstream quality-related research, a reduction of £40m for teaching (including a reduction in PGT FTE funding rate), maintaining the disabled students premium at the 2016/17 level, the inclusion of nursing, midwifery and allied health professions (£32m), cuts to the student premium budget for full time UG of £20m (part time UG funding remains static). Institutions will receive individual allocations in April although with a publication embargo in force until May. Capital allocations will be announced in March.

Student Loans Sale: A parliamentary question tabled by Steve McCabe requested publication of the ‘in-depth market testing exercise associated with the same of the student loan book. Jo Johnson has responded: “The Government ran a market testing process with a cross-section of potential investors in the student loan book from the end of September into November 2016. This sought feedback on potential sale structures and key features of the transaction and informed the design of the sale. This was a commercial rather than a public process and was conducted under non-disclosure agreements. We do not intend to publish a report of the details. Protecting the details of the conclusions of market testing will help the ongoing sale process achieve value for money for taxpayers.”

Student Fees: On Thursday 16th the Petitions Committee released its latest decisions regarding recent petitions with a high number of signatures. This included a petition to government to change the University fees from £9250 back to the £3000 fee. The Committee agreed to wait for the Higher Education and Research Bill to complete its passage through Parliament before deciding whether to schedule a debate – effectively this was a dismissal of the petition.

Research Excellence Framework  The responses to the REF2021 consultation were due in by midday on 17th March.

  • There has been a lot of focus on one area, the definition of “research active staff” for the returns – there are some interesting views:
  • HEFCE blog (and BU’s reply) – HEFCE are proposing a negotiated definition for each university, BU is proposing all staff should be returned, including teaching only
  • Royal Society blog on Research Professional – they say staff shouldn’t be returned at all, it should be institutional
  • The PVC (Research and Enterprise) from Hertfordshire says on Times Higher Education that the solution is flawed and that clarity is needed

There are many other issues in the REF consultation, including the portability of outputs, which will have important consequences for institutions and their staff. The HEFCE REF consultation on the implementation of the REF 2021 closed on 17 March 2017.  You can read BU’s response here.

Blog by the Vice-Chancellor – what next for the Teaching Excellence Framework

Fusion, policy, Research assessment, Research news, Research Training, RKE development framework, Uncategorized

The BBC 2 series “Meet the Lords” could not have been better timed. The House of Lords has flexed its muscles on the Article 50 Bill and this week’s episode coincided with them passing an amendment to the Higher Education and Research Bill (HE Bill) that breaks the link between the Teaching Excellence Framework (TEF) and fees. Since then another amendment has been passed that would change the nature of the TEF, and bring it under Parliamentary scrutiny.

It would be easy to dismiss these (as some have done) as acts of rebellion by a non-elected chamber that is in the case of the HE Bill, representing vested interests in the face of a genuine government attempt to reform a sector that is badly in need of it. The Department for Education could be forgiven if they had thought that the HE Bill was nearly home and dry. They had published a long list of amendments which had been largely welcomed by the sector. The TEF does not require Parliamentary approval. Universities UK and GuildHE, amongst others, had expressed support for the HE bill as amended and expressed support for the TEF – opposing the addition of more detail as it would reduce flexibility in future negotiations on the detail. But the House of Lords did not agree – they have not sought to add more detail in the TEF, but to change its nature completely. Reading the debates, it is clear that members of the House of Lords, like most of the sector, generally support the objectives of TEF in bringing focus on the quality of education and student outcomes. They support the provision of more and better information about universities for applicants and others. They, like many in the sector, also generally support an inflationary increase in fees.

In the latest amendment, the provisions for the TEF in clause 26 have been removed and the new clause instead requires the Secretary of State to bring forward a scheme to identify whether an institution meets or fails to meet expectations based on quality standards but it “must not be used to create a single composite ranking of English higher education providers”.  The arguments are neatly summarised by Lord Lucas: “Bronze will be seen as failing because these universities will be marked out as the bottom 20%. This is just not necessary. We have succeeded, in our research rankings, in producing a measure of sufficient detail and sophistication for people to read it in detail. It produces quite marked differences between institutions, but nobody reads it as a mark of a failing institution. It is information, not ranking…”.

An earlier amendment removed the differentiation between fees based on different ratings. The speeches in the House of Lords demonstrate that they are opposed to this link for different reasons, for example:

  • Baroness Deech “If we detach fees from gold, silver and bronze, we stand a chance of increasing social mobility under the amendment. If we do not, social mobility will be frozen and ghettoisation will increase.”
  • Baroness Wolf of Dulwich: “I want to cite three groups of academics ….all of which feel, as do students, that in their current state the TEF metrics are not up to the job of determining fee levels and that, until we are sure that we have valid and reliable measures, we should not do this.”
  • Lord Lipsey : “… what seems knocking on bizarre is to plough on with bringing in this link between fees and the TEF before we have got the TEF right….The Government would give themselves the best chance of proving themselves right and the sceptics wrong if they gave time for the TEF to settle down before they brought in the fees link.”
  • Lord Kerslake: “My second reason for not making the link is that the TEF rating will relate to the university, not the subject or course. We will not see subject-level ratings until 2020 and yet we know that it is perfectly possible to have a mediocre course in an otherwise excellent university, and indeed vice versa. It can be argued that the TEF ranking gives an indication of the overall ​student experience at a particular institution, but the variation which so obviously exists within institutions makes that argument quite unconvincing.”

Except for the subject level fee point (which has not become a topic of debate yet), these are all arguments that were made by the sector in responding to the Green Paper and the TEF consultation. These are all things that we have continued to raise as we discuss the implications of subject-level TEF.

So as it stands, the TEF has lost both of its “incentives” – aka its carrot and its stick, which were both in the form of the impact on fees and reputation. It is not at all clear what will happen next – some ideas are given in this Wonkhe blog. In blogs on the Times Higher Education, Maddaleine Ansell of the University Alliance and Sorana Vieru gave very different perspectives.

So what compromise could there be to address all the concerns and yet still preserve the positive aspects of the TEF – i.e. the increasing focus on education and outcomes? I go back to BU’s response to the Green Paper, when we said that the TEF should model itself on the REF.. It should celebrate excellence wherever it is found, there should not be a link with tuition fees and there should be no forced ranking. To achieve that now, a remodelled TEF could include the following features:

  • no link to fees
  • have two rather than three levels of award – perhaps indicating good and outstanding. The last category is those who fail their quality assessment and don’t qualify for TEF.
  • take a different approach to benchmarking that does not force differentiation
  • include a place for commendations

I am not convinced by the argument that no-one would participate in the TEF without the direct financial incentive. That does not hold true for the REF. The REF has increased the focus on impact and had a beneficial impact on research. (We have some reservations about the changes proposed in the latest REF consultation, but that is a separate issue.) The concerns about the TEF would be mitigated substantially if the Olympic rating system and the link to fees were dropped. The sector would be able to engage in a much more constructive debate about subject-level TEF.

The TEF does not need to be thrown out completely – but this is an opportunity to go back to where this started from and ensure that the TEF brings focus on the quality of education and student outcomes.

HE policy update w/e 24th February 2017

innovation, international, policy, Research news, Research Training, RKE development framework, Student Engagement, Training

Jo Johnson spoke at a UUK conference today and made a number of important announcements:

  • New government amendments to the Higher Education and Research Bill. The detailed amendments have not yet been published but a Department for Education factsheet has been provided. The government amendments have been welcomed so far. See the Latest set of proposed amendments (mostly opposition amendments but some government ones too) – this will clearly grow more before the report stage starts on 6th March 2017. See more below.
  • Importantly, he announced that the subject level TEF would have a two year pilot – starting in 2017/18 but also running through 2018/19. Subject level TEF would then be formally implemented in 2019/20, with ratings that are announced around May 2020 with the Year 5 institutional level ratings. Note that it is currently not intended that subject level TEF will result in subject level fees. There was no mention of TEF for post-graduate, which was originally planned to run in year 4, so assessed during 2018/19.
  • Accelerated degrees – Jo Johnson also wrote in the Times about the government response to the consultation on accelerated degrees and credit switching (that closed last July with 1000s of responses) which will be issued shortly, and relevant changes that will be made to the HE Bill.  Apart from the headline grabbing focus on universities being able to raise fees above £13,000 a year, this consultation response will probably contain interesting stuff on credit transfers between universities.  The headline focus on fees is a little bit misleading, because this is in response to sector feedback that it isn’t possible to provide three years of teaching in two years unless fees are increased for those two years (there were many other comments about the impact on extra- and co-curricular activities, as well as cost).  The higher fees would only apply to accelerated degrees, as The Times story makes clear.
  • He also announced a number of other changes regarding institutional and research autonomy which are very helpful – more detail is given below.

HE and Research Bill – As mentioned above, the amendments continue to accumulate for the Lords report stage of the HE bill with the latest government amendments yet to be published. See the latest round up from Wonkhe here. One joint government and Labour amendment (to replace the opposition amendment passed in the House of Lords) defines institutional autonomy, and a number of others require the OfS to protect that autonomy. The definition that is proposed is set out below:

  • There is a new transparency duty – one that did seem to be an omission in the previous drafting. The TEF reflects the new focus on widening participation away from just access to progression and outcomes, but the HE Bill did not reflect this fully in the transparency duty as originally proposed. This has now been picked up, and the proposed amendment (which we have not seen yet) will require providers to publish information on levels of attainment, in addition to application, offer, acceptance and completion rates, broken down by gender, ethnicity and socio-economic background.
  • After the long and energetic discussions in the Lords about the Haldane principle, research autonomy and the many concerns expressed about the role of the Secretary of State, a number of amendments have been tabled 9but not yet published) including one explicitly about the Haldane principle
  • There is a response to the criticism of the limitations on the role of Innovation UK and some of its financial arrangements – we don’t have the detail on all of it but the business focus amendment is set out in the explanatory paper
  • As noted above, the announcement on accelerated degrees requires a change to the HE Bill to allow for higher fees for accelerated degrees – the DfE paper is clear that the overall cost of a degree will not go up – but universities will be allowed to charge more per year for the more intensive short courses.
  • On credit transfer, the proposed amendments will apparently require the OfS to monitor and report on arrangements for student transfer and a power for them to encourage and promote it.
  • There are a number of more technical amendments proposed, including to protect institutions in cases where degree awarding powers may be revoked, to protect Royal Charters and to ensure that the OfS does not meddle in institutional autonomy as regards standards. These changes will be most welcome, and BU, along with most of the sector has called for these changes, and we are looking forward to seeing the detail.

Other proposed opposition amendments include:

  • yet another attempt to change the name of the OfS – this time to the Office for Higher Education Standards. Given that the government have just confirmed that the OfS should not meddle in standards (see above), this amendment seems unlikely to pass.
  • and another attempt to address student loan repayment terms and conditions – this has been raised at every opportunity so far but has not yet been subject to a vote.

The HEFCE grant letter is out, with extensive coverage.  Research Professional report that:

  • Teaching funding will fall, representing nine consecutive years of reductions- it is due to be cut by 5 per cent in 2018-19—and the funds now also have to cover the expansion of medical schools and include trainee nurses, midwives and other health professionals. This is particularly interesting because of the theory that removing the commissioning arrangement will increase student numbers, balanced against concerns in the sector about falling applications and the real-life challenge with increasing student numbers, i.e. placements – On the latter point, Research Professional note the part of the HEFCE letter that “adds that in order to implement the Department of Health announcement that, from September 2018, the government would fund up to 1,500 additional student places in medical schools each year, the funding council should make an initial allocation of 500 places in 2018-19 “based on the capacity for growth and viability of provision in different institutions and be informed by advice from the Department of Health and Health Education England on the distribution of medical placements”
  • On PG: “The letter further asks that the funding council ensures it pays for postgraduate courses on a basis that is “consistent with and complementary to” the new postgraduate loan system. In particular the funding council is asked to prioritise science and other high cost subjects.”
  • On TEF: “It asks the funding council to continue funding the introduction of the teaching excellence framework, including the subject-level pilots, for which the budget and approach will be determined later in 2017. The letter further encourages the continued funding of the “high priority” learning gain pilots” – which may give us an idea of where TEF metrics may be headed
  • On schools: “On social mobility, government asks the funding council to continue encouraging “innovative” forms of engagement with schools, including work to identify which institutions are sponsoring or establishing schools and the support they require to achieve it.” This sounds like a slight backing away from compulsory sponsorship of schools? Or maybe just preparation for identifying those who don’t comply.
  • On research:  “the letter says that detailed allocations for the £4.7 billion of additional investment pledged in the autumn statement will be finalised in early March. The funding council is expected to distribute the additional research and knowledge exchange funding allocation in 2017-18. The £100 million announced for technology transfer, meanwhile, will be based on competitive mechanisms that the funding council will develop with the Department for Business, Energy and Industrial Strategy.”
  • And on student wellbeing: “The letter further asks the funding council to implement recommendations made by the Universities UK taskforce on violence, sexual harassment and hate crimes, which advocated the embedding of a zero-tolerance culture towards such incidents on campus. It also recommends the creation of an evidence base around mental health needs and services for staff and students.” The latter cross refers to a UUK good practice guide.
  • Other issues include plagiarism, credit transfer, REF, degree apprenticeships, Prevent and efficiency.

BTEC students – the Higher Education Policy Institute (HEPI) have issued a report on BTECs and Pearson have blogged for them.  The report concludes that “Students arriving at university with BTECs account for much of the growth in students from the lowest participation neighbourhoods and other under-represented groups over the past decade. But those with BTECs face a ‘glass ceiling’ – for example:

  • only 15 BTEC students were accepted at the four most selective higher education institutions in 2015; and
  • under 60 per cent of students with BTECs at Russell Group universities complete their course.”

The report makes the following policy recommendations:

  • As the proportion of pupils achieving the highest BTEC grades (equivalent to three A-Levels) more than doubled from 17% to 38% between 2006 and 2013, the Government should evaluate whether the current system of external verification of BTECs is fit for purpose.
  • Universities should issue collective guidance on which BTECs are most valuable to students in terms of progression, as they have already done for A-Levels.
  • More prestigious universities with low numbers of BTEC students should consider bespoke access courses for BTEC students aimed at helping them adjust to the methods of teaching and assessment that are common in higher education.

Lifelong learning – The University Alliance issued a report on lifelong learning which calls for a number of actions, including setting up a UCAS style system for adult learning courses, reintroducing individual learning accounts and providing additional loans. They also mention accelerated degrees and suggest broadening the apprenticeship levy to cover such course.

Essay Mills – Jo Johnson called this week on universities to do more to stop students buying custom written essays online. He has asked the QAA to prepare guidance for universities and information for students to help combat the use of ‘essay mills’ websites as well as other forms of plagiarism and for the QAA to take direct action against those marketing the services. It looks as if the guidance will focus on making sure that universities have policies and sanctions in place.

“The Universities Minister has asked for guidance aimed at universities and information for students to help combat the use of these websites, as well as other forms of plagiarism. The Quality Assurance Agency (QAA) has also been tasked to take action against the online advertising of these services and to work with international agencies to deal with this problem.

The Minister is calling for the guidance to include tough new penalties for those who make use of essay mills websites, as well as the need to educate students about the potentially significant negative impacts on their future career if they are caught cheating.

Universities Minister Jo Johnson said: “This form of cheating is unacceptable and every university should have strong policies and sanctions in place to detect and deal with it“. Essay mill websites threaten to undermine the high quality reputation of a UK degree so it is vital that the sector works together to address this in a consistent and robust way.””

Brexit – The Department for Business, Energy and Industrial Strategy (BEIS) has published the terms of reference and membership of its “high level stakeholder working group on EU exit, universities, research and innovation”.

“The overarching purpose of the group is to provide a forum for BEIS, DfE, DExEU, and a broad range of UK representatives of the universities, science, research and innovation communities to discuss issues of common interest in approaching the UK’s exit from the EU. The emphasis will be on considering all factors related to research and innovation that need to be taken into account as government policy develops.”

Membership includes Jo Johnson, Madeleine Atkins, Nicola Dandridge, and the chairs of Million Plus, the Russell Group, GuildHE and the University Alliance, a couple of VCs and PVCs from the devolved administrations (Heriot-Watt, Ulster, Cardiff)  and representatives of a number of science bodies such as the Royal Academy of Engineering, the Campaign for Science and Engineering and others, CBI, RCUK, UKRI.  And Sir Mark Walport in his current role as the Government Chief Scientific Adviser.  Interestingly, none of the bodies represented are arts or social sciences bodies, which continues to demonstrate the apparent assumption in all of these groups on research as an activity that is only relevant to STEM.

Competition and Markets Authority (CMA) have intervened following a complaint to OIA from students that didn’t receive sufficient notice that compulsory modules had been introduced to their course. While UEA was not considered by OIA to have breached its rules the CMA have asked UEA to change its policy and consider the introduction of compulsory modules as a substantial change which would require greater timeliness of notice in future. CMA’s intervention is seen as a landmark intrusion by some. The CMA request is discussed further by Jim Dickinson and Paul Greatrix on Wonkhe. UUK also have a blog on the subject.

HEFCE consultation on REF2021 – what you need to know

Research assessment

ref-logoIt hardly feels like the dust has settled on the results from the last Research Excellence Framework, yet here we are again – almost half way through another research assessment cycle with HEFCE proposing potential changes to the next exercise. It feels like I’ve been transported back to 2009 when HEFCE were running bibliometrics and impact pilots to inform REF2014. I have to remind myself that it is in fact 2017 and things have moved on. We are now talking about REF2021 (how time flies!) and HEFCE have launched a consultation on how this exercise might work. Whilst citations and impact are still on the agenda for discussion, they are joined by other substantial changes that, if adopted, will transform the shape and potentially outcome of the next exercise.

The proposed changes have been informed by Lord Stern’s independent review of the REF which was published last July. This made 12 recommendations on how the next REF could be strengthened, whilst reducing the bureaucracy and cost of running the exercise (read Jane Forster’s overview of the recommendations – Stern review of the REF: what next?). These recommendations have now been reviewed by the four UK HE funding bodies and, prior to Christmas, HEFCE published proposals to amend the REF to incorporate the recommendations. HEFCE’s proposals are now open for consultation with the sector.

The document itself is fairly lengthy and dense, and over 40 questions are put forward for consideration. The key proposals are:

  • All research-active staff to be submitted
  • The decoupling of staff from outputs
  • Outputs will no longer be portable across institutions
  • All outputs must be available in open access form (with some exceptions)
  • Impact will have a broader definition
  • Institutional environment narratives and case studies to be submitted

There are some excellent summaries available online, many of which provide thoughts on what the proposals could mean in practice. The following summaries are particularly informative and worth reading: HEFCE launches consultation on REF2021, Soft Stern or Hard Stern and Implementing REF2021.

BU will be submitting an institutional response to the consultation before the deadline of 17th March. As these are significant proposals, all staff will have the opportunity to contribute to the response. As a number of the key questions would benefit from a discipline-level response, UoA teams (UoA leaders, impact champions and output champions) are calling meetings within Faculties to hold discussions related to their own UoA. These will take place over the next month or so. Contact a member of your UoA team for further information. Staff not based in Faculties are also invited to comment on the proposals and can do so via their Director/Head of Professional Service.

The are various REF-focused conferences and events taking place over the next few months, organised by HEFCE, ARMA, etc. Interesting news from these will be posted to the Research Blog. HEFCE are anticipating significant sector-wide engagement with the proposals and have committed to reading all feedback received. It is anticipated that, following the consultation, the initial decisions regarding the shape of REF2021 will be published in summer 2017.

From Draft to REF with CEMP’s new Publishing Partnership Initiative

BU research, Centre for Excellence in Learning, Fusion, PG research, REF Subjects, Research Training

Do you have an incomplete paper that you’ve been sitting and can’t seem to finish? Have you recently presented at a conference, but haven’t written-up a paper out of your presentation yet? Did you get a rejection and are struggling to get motivated again? Or maybe you’ve written a couple chapters of your PhD and are hoping to turn one into a publication?

Whatever the scenario, CEMP’s new Publishing Partnership Initiative (PPI) can help you to collaborate with another researcher to turn your ideas into a viable REF-ready journal article. And did we mention, you can win a free dinner for two?

Publishing Partnership Workshop
Thursday January 26th
Talbot Campus, BU
11:00-14:00
WG 05 (Weymouth House)

To launch the initiative, this workshop will introduce the Publishing Partnership Initiative and help you find a good match to develop your REF output. We will also discuss strategies and tips for working toward REF submissions:

11:00-11:15 Introductions and coffee

11:15-11:30 Welcome to the PPI scheme (Anna and Richard)

11:30-12:00 Hear No Evil, See No Evil: What you need to know about REF and the Sterne review (Julian & Dan J)

12:00-12:45 Interactive session: Strategies for fitting writing into our busy schedules (Brad & Karen)

12:45-13:15 Academic Match.com: Finding the right journal and writing partner for your research output (Anna and Isa)

13:15-14:00 Catered planning lunch with our publishing partners

To participate: Send a 500+ word rough draft or outline of a potential research paper to afeigenbaum@bournemouth.ac.uk AND Richard rberger@bournemouth.ac.uk by Monday 23rd January 2017. This might be an abandoned draft, a conference version of a paper presentation or an outline for a possible research paper. At this stage, any draft you have might be gold, so don’t be shy!

To be a mentor or writing partner: Send an email with a list of your research and methodological areas of expertise to afeigenbaum@bournemouth.ac.uk AND Richard rberger@bournemouth.ac.uk by Monday 23rd January 2017.

 

Eat your Success! Partners who successfully submit a paper to a peer review journal within the 5 month time frame will receive a ‘dinner for two’ voucher to celebrate their success.*

*Voucher is worth up to £45.00. Does not cover alcoholic beverages.

We will aim to pair colleagues around expertise either in the same research area or in relation to the methodological approach. At least one partner will be accustomed to journal publishing and the REF process, as well as to the challenges facing us to find the time to research and write. In the first instance, this workshop is geared toward UoA 25, 34 and 36 entries (covering Education, Media Practice and Media Studies). For full details see: http://www.civicmedia.io/events-2/publishing-partnership-initiative/

*This project was initiated by Anna Feigenbaum and Richard Berger. It is supported by the UoA 25 development fund, CEMP, the Civic Media Hub & the Journalism Research Group.

How to Write a 4* Article

BRIAN, BU research, Guidance, Publishing, REF Subjects, Research assessment, Training, writing

Prof. Mark Reed

A fortnight ago Prof Mark Reed, Professor of Socio-Technical Innovation at Newcastle University and the man behind Fast Track Impact, tweeted some thoughts on how to write a 4* paper for the REF. He went on to explain his thinking in more detail in a guest post on the Research Fundementals blog, the post is published here with the authors permission.

_____________
How do you write a 4* paper for the Research Excellence Framework (REF)? It is a question I’ve asked myself with some urgency since the Stern Review shredded my REF submission by not allowing me to bring my papers with me this year to my new position at Newcastle University.

Obviously the answer is going to differ depending on your discipline, but I think there are a few simple things that everyone can do to maximize their chances of getting a top graded research output.

I’m going to start with the assumption that you’ve actually done original, significant and rigorous work – if you haven’t then there is no point in reading any further. However, as I am increasingly asked to pre-review papers for colleagues across a range of disciplines, I am seeing examples of people who write up work as a 2* or 3* paper that has the potential to get a better score. I should point out that I believe that there is an important role for 1* and 2* papers, and that I regularly write these on purpose to address a problem of national significance and frame it for the specific, narrow audience that is likely to be able to benefit most from my work. However, whether I like it or not, as a Professor in a research-intensive University, there is an expectation that I will be submitted as a 4* researcher, which means I need a few 4* papers as well.

You can see some more detailed thoughts on what I think makes 4* for different types of paper in this Tweet:

As you’ll see from the discussion under that tweet though, my more detailed thoughts probably only apply to Units of Assessment across panels A-C, and probably isn’t relevant to the arts and humanities.

Having said this, I think there are a number of things we can all do to maximize the chances of our work being viewed favourably by REF panelists.

  1. Write to the criteria: when I was learning to drive, my instructor told me that in the test I should make sure I moved my head when I was looking in the rear view mirror, to make sure the examiner noticed I was using my mirrors. We’re all used to writing to the criteria of funding calls, and in fact we are all perfectly used to writing papers to the criteria of our target journals. In the last REF, research outputs were judged against three criteria: originality, significance and rigour. Whatever the interpretation of these criteria in your discipline, have you made it explicit to REF panelists reading your work exactly what is original, and why it is so original? Have you explained and effectively justified the significance of your work? And have you included evidence that your methods, analysis and interpretation is rigorous, even if you have to use supplementary material to include extra detail about your methods and data to get around journal word limits?
  1. Get REF feedback before you submit your work for publication: find out who is going to be reviewing research outputs for REF internally within your Unit of Assessment at your institution and ask them to review your work before you submit it. They may be able to make recommendations about how you might improve the paper in light of the REF criteria. Sometimes a little bit of extra work on the framing of your research in relation to wider contexts and issues can help articulate the significance of your work, and with additional reading and thinking, you may be able to position your work more effectively in relation to previous work to demonstrate its originality more clearly. Adding a few extra details to your methods and results may re-assure readers and reviewers that your approach is indeed rigorous. This is not just about doing world-leading research; it is about demonstrating to the world that your work is indeed world-leading. For me, these criteria are nothing new and are worth paying attention to, whether or not we are interested in REF. Meeting these three criteria will increase the chances that you get through peer-review and will increase the likelihood that your work gets cited.
  1. Analyse and discuss good practice in your own area: the only way to really “get your eye in” for REF is to actually look at examples of good and poor practice in your own area. Below, I’ve described how you can design an exercise to do this with your colleagues. You can do it yourself and learn a lot, but from my own experience, you learn a lot more by doing this as a discussion exercise with colleagues who work in your area. If you can, take notes from your discussion and try and distill some of the key lessons, so you can learn collectively as a group and more effectively review and support each others’ work.

How to organize a discussion to work out what makes a 4* paper in your area:

  • Identify top scoring institutions for your Unit of Assessment (UOA): download the REF2014 results, filter for your UOA (columns E or F), then filter so it only shows you the outputs (column J), and then filter for 4* (column L), showing only the institutions from your UOA that had the highest percentage of 4* outputs. Now for those institutions, look across the table (columns L-P) to see which has the highest proportion of outputs at either 3* or 4*. For example, an institution may have 80% of its outputs graded at 4* and 15% graded at 3*, meaning that 95% of its outputs were graded at 3-4*
  • Download a selection of papers from the top scoring institutions: go to your UOA on the REF website, find and click on the institutions you’ve identified in step 1, under “view submission data”, click on “research outputs”, copy and paste output titles into Google Scholar (or your search engine of choice) and download the articles. You may want to select outputs randomly, or you may want to go through more selectively, identifying outputs that are close to the areas your group specialize in
  • Repeat for low scoring institutions so you can compare and contrast high and low scoring outputs
  • Discuss examples: print copies of the high and low scoring outputs, labeled clearly, and in your next UOA meeting, let everyone choose a high and a low-scoring example. Given them 10-15 minutes to quickly read the outputs (focusing on title, abstract, introduction, figures and conclusions so you’re not there all day) and then ask the group (or small groups if there are many of you) to discuss the key factors that they think distinguish between high and low scoring outputs. Get your group(s) to distill the key principles that they think are most useful and disseminate these more widely to the group, so that anyone who wasn’t present can benefit.

It would be great if I could tell you that these are my “three easy ways to get a 4* paper” but doing work that is genuinely original, significant and rigorous is far from easy. If you have done work that is of the highest quality though, I hope that the ideas I’ve suggested here will help you get the credit you deserve for the great research you’ve done.