BU Research Blog

There so much regulatory stuff to talk about, so we are focussing on that this week.  There will be a more normal update next week.

Moving the goalposts  – the OfS proposals for regulating absolute baselines

This is the biggy – the one with the absolute student outcomes metrics.  The 60% metric that was all over the news is for FT first degree undergraduates using the “progression” metric – further study or professional or managerial employment.  But there are continuation and completion baselines too, and they vary dramatically for PG students and a whole bunch of other categories.

These become binding licence conditions, and breaches of them lead to serious regulatory consequences.  Serious breaches could lead to losing degree awarding powers, or not being allowed a TEF rating (and having fees capped at £7500 as a result).  There are lots of lesser actions, including specific licence conditions requiring action to address things, as well.

And the data will be published.  All of it.  Including a lot of split data.  Potentially 48 indicators, all with spits by age, disability, sex, ethnicity, IMD, etc etc…and SUBJECT.  You will recall that there is no subject level TEF.  While that is true, the regulatory baseline data, and the TEF data, will be presented by subject because the OfS want to be able to identify “pockets of poor provision”.  Having these pockets could cause a regulatory problem, and drag a provider down in the TEF.  The data will include taught students, registered students, and “taught and registered”.  So any partner students would count towards our data as well as the partner’s data. It will be based on 4 years of data.

And how will this all work?  There will be an initial review of the first lot of data  – this autumn – and then an annual cycle.  There will be intervention letters for those with problems in October 2022.  The data already exists, so no need to wait.  They are consulting on how to prioritise the challenges – not everyone with a breach in a small pocket will get one.  They will look at context – a bit, but note:

• We do not consider that a provider’s mission or strategy is relevant to our consideration of whether it is delivering positive outcomes for students 
• We do not consider that the level of funding a provider receives is relevant to whether it should meet the minimum requirements set by the OfS 
• We do not agree that lower entry tariffs should be a reason for performance below our minimum requirements 
• We do not consider that a provider with New DAPs should be exempt from satisfying minimum regulatory requirements 
• We do not consider that a provider’s resources are relevant to whether it should be expected to meet the OfS’s minimum requirements 
• We do not consider that a provider’s reputation is relevant to our consideration of whether it is achieving positive outcomes for students 

Worried about regulatory burden – too bad.  They think it is proportionate.  And they aren’t telling us to change our internal monitoring to look at all this.  But we might want to – and it has the potential to be very onerous indeed.

Supporting documents:

• detailed guide to how the numerical thresholds have been set
• analysis of responses on the phase 1 consultation from November 2020 that relate to regulation of student outcomes

Wonkhe have coverage in an article by Jim Dickinson which is worth reading.

Teaching Excellence Framework – it’s ALIVE

The TEF had become a zombie – the walking dead scheme where awards were still in force (because apart from anything else they are required as a licence condition) but shhh, providers aren’t allowed to talk about them because they are so out of date as to be potentially seriously misleading.  And the OfS has been talking about a new TEF for a very, very long time.  And finally, linked very closely to the new absolute baselines but with some exciting new bits as well, here it is.  Hold on to your hats, it is going to be a busy summer and autumn, especially as we won’t get the guidance until June!  They were already given feedback about timing in their consultation events and this is therefore probably the best we are going to get.  Submissions by mid-November 2022.

So it’s back, bigger and better.  Still called the TEF (not the TESOF).  Still gold, silver and bronze.  There is a new category of “requires improvement” for those who don’t get one-  they are asking for input on how to communicate that one.  As noted above, institutional level only, although we get (and have to address) all the subject level data.  There is an institutional submission that will be 20 pages, and a separate student one of 10 pages.  It will happen every 4 years, dropping the annual cycle we had before, when you could try again for a higher grade.  The data will be published annually though.

It will still use NSS, as well as the three outcomes measures referred to above (professional employment or further study, completion and continuation).

The “aspects” of measurement are new: student experience (academic experience and assessment, and resources, support and student engagement) and student outcomes (positive outcomes, as above, plus a whole new one – educational gains).  So far so familiar, but the educational gains is fascinating.  No data for this one – “these features should relate to a provider’s articulation of the gains it intends its students to achieve; its approach to supporting these educational gains; and evidence of the gains achieved”.

And the much challenged benchmarking is different (can’t say yet if it is better) and there are no “flags”.  Just a very complex graphical representation. One thing that veterans of the last process will be pleased to hear is that providers will receive the initial conclusion and evidence and have 28 days to make representations.

Supporting documents:

• Materiality and high benchmark values to use in interpretation – not yet published

Wonkhe cover this one too, in an article by David Kernohan: Frankly, it’s better than it has been in the past, but still probably not as good as it could have been. It’s certainly better than the B3 proposals.

Research Professional also have an article and it featured in the 8am Playbook too.

And last, but definitely by no means least

All of these things need data.  There is a separate 195 page consultation on how the data for the B3 conditions and the TEF will be calculated.  Along with all of you, we look forward to working out the detailed implications of all of that.  Consultation on constructing student outcome and experience indicators for use in OfS regulation (with a video)

• Comparison of completion methods
• Technical documents
  • Description and methodology
  • Core algorithms
  • Subject code mappings
  • Instructions for rebuilding the datasets
  • Description of statistical methods
• Data dashboards (illustrative and using fictional data)

If nothing else, please have a look at the dashboards.  They show the new world in glorious technicolour.

Other changes – the proposed licence conditions you might have missed

There was a previous consultation over the summer (closed at the end of September) that set out the other new proposed B conditions.  This one had an ambitious implementation timeline which has not stuck.  What it said was “we intend to make a decision on whether to impose the conditions in Annexes A, B and C and revise the regulatory framework, as set out in these proposals, in autumn 2021. The new ongoing conditions would come into effect for registered providers on the date of publication of that decision“.  Assuming that they don’t make any changes to what was proposed (and they don’t make many, usually), this lot could be imposed imminently.  Or, the delay may be because they are revising it substantially.  Either way, a  bit of notice would be helpful.  Not because we aren’t doing this stuff, but because we need to make sure we have the monitoring and audit trails in place to prove it if we are asked.  Which we could be, either as part of a review linked to the outcomes data, or because other OfS monitoring suggests that there may be an issue with some of these.

Yesterday’s announcements referred to the fact that this previous consultation had happened, but said nothing about next steps on that.  So maybe it will all come together in June.  Or we might get the first lot earlier.  So what are they?  You need to know, because if implemented as proposed, they are very wide ranging.

B1 …. the provider must ensure that the students registered on each higher education course receive a high quality academic experience.   

B1.3 For the purposes of this condition, a high quality academic experience includes but is not limited to ensuring all of the following: – that each higher education course:

1. a. is up-to-date;  
2. provides educational challenge;  
3. is coherent;  
4. is effectively delivered; and
5. as appropriate to the subject matter of the course, requires students to develop relevant skills.  

Those highlighted definitions are worth looking at.  There is a lot more about these in the guidance which is summarised in the attachment. One question is how the OfS will assess all these things – see below.  And of course there are lots of other questions – if the rumoured next steps on Augar are true, which is a push to modular learning to support the lifetime learning objective, how does that fit with “coherence”.  And just note – students and courses include PGR programmes of study.

“up-to-date” means representative of current thinking and practices in the subject matter to which the HE course relates…

“educational challenge” means a challenge that is no less than the minimum level of rigour and difficulty reasonably expected of the HE course, in the context of the subject matter of the course [this seems circular to me!]

“coherent” means a HE course which ensures:

1. there is an appropriate balance between breadth and depth of content;
2. subjects and skills are taught in an appropriate order and, where necessary, build on each other throughout the course; and

iii. key concepts are introduced at the appropriate point in the course content.

“effectively delivered”, ….means the manner in which it is taught, supervised and assessed (both in person and remotely) including, but not limited to, ensuring:

1. an appropriate balance between lectures, seminars, group work and practical study, as relevant to the content of the course; and
2. an appropriate balance between directed and independent study or research, as relevant to the level of the course.

“relevant skills” means:

1. knowledge and understanding relevant to the subject matter and level …; and
2. other skills relevant to the subject matter and level …including, but not limited to, cognitive skills, practical skills, transferable skills and professional competences.

B2.  ….the provider must ensure:  ..each cohort of students registered on each HE course receives resources and support and effective engagement with each cohort of students, in both cases to ensure:  

1. a high quality academic experience for those students; and
2. those students succeeding in and beyond higher education; and

“resources” includes but is not limited to:

1. the staff team … being collectively sufficient in number, appropriately qualified and deployed effectively to deliver in practice; and
2. physical and digital learning resources that are adequate and deployed effectively to meet the needs of the cohort of students.

“support” means the effective deployment of assistance, as appropriate to the content of the HE course and the cohort of students, including but not limited to:

1. academic support relating to the content of the HE course;
2. support needed to underpin successful physical and digital learning and teaching;
3. support relating to avoiding academic misconduct; and
4. careers support, but for the avoidance of doubt, does not include other categories of non-academic support.

“engagement” means routinely building into the course delivery opportunities for students to contribute to the future development of the HE course in a way that maintains the academic rigour of that course…. 

Other new conditions: As the summary document sets out, there are conditions on assessment, the credibility of awards, and lots more, including the requirement to keep copies of assessed work so that the credibility and reliability of awards can be assessed by the OfS if it needs to be.

Subscribe!

To subscribe to the weekly policy update simply email policy@bournemouth.ac.uk. A BU email address is required to subscribe.

External readers: Thank you to our external readers who enjoy our policy updates. Not all our content is accessible to external readers, but you can continue to read our updates which omit the restricted content on the policy pages of the BU Research Blog – here’s the link.

Did you know? You can catch up on previous versions of the policy update on BU’s intranet pages here. Some links require access to a BU account- BU staff not able to click through to an external link should contact eresourceshelp@bournemouth.ac.uk for further assistance.

JANE FORSTER                                            |                       SARAH CARTER

VC’s Policy Advisor                                                              Policy & Public Affairs Officer

Follow: @PolicyBU on Twitter                    |                       policy@bournemouth.ac.uk