BU Research Blog

We focus on the interesting set of reports released as people clear their desks before their summer holidays.

Learning gain

The OfS have published evaluations of the various learning gain projects that have been running for some time.  The OfS website is very clear “The report below is independent research which we have commissioned. As such, it does not necessarily reflect the views or official position of the OfS.”  You will recall that one reason for HEFCE setting up these projects was to see if a learning gain measure could be created for the TEF.  The answer would seem to be “no” although given the disclaimer, that might not stop them having a go.

On the final evaluation of 13 projects, the conclusions are:

• Learning gain can be defined as the change in knowledge, skills, work-readiness and personal development, as well as enhancement of specific practices and outcomes in defined disciplinary and institutional contexts.
• embedding measures in curriculum design is the most effective approach for collecting data for measuring learning gain
• Pilots of standardised tests carried out during the projects have not proven to be robust and effective measures of learning gain due to challenges of student engagement, differential scores across socio-demographic characteristics, subject differences and use of data.
• Contextual factors such as subject-level differences, institution type and student characteristics differences impact the transferability of measures of learning gain. These differences should be considered when designing and selecting learning gain measures; when analysing and presenting findings. Mediating effects need to be considered.

The report on the National Mixed Methods project  has the following recommendations:

For policy-makers and providers      

• A one-size-fits-all measure of learning gain (modelled on the NMMLGP questionnaire) should be abandoned as it holds minimal value for the majority of students and is not an influential construct in their present decision-making concerning either choice of institution or impact on the curriculum.
• Students’ perceptions of learning gain need further exploration in order to move beyond what are acknowledged as impressionistic findings reported here, which are bound by proportionality constraints.
• The sector needs to consider whose interests are best served by the measurement of learning gain. The evidence gathered here from participating providers and their students indicates that there is a dichotomous view of learning gain: either as a marker of institution positioning within a market-oriented system; or as a process of progression throughout the student journey. The two things are not necessarily synonymous.

For higher education providers

• All learning gain work needs to be related to students’ own context and clearly embedded at local level within the subject or disciplinary area. Engagement is also highly dependent on whether any initiatives are promoted by trusted sources such as course tutors, rather than unfamiliar contacts.
• Providers should consider developing a repertoire of approaches, as part of a learning gain toolkit, which can be accessed by students as part of a flexible and adaptable process underpinned by student choice rather than normative comparison. Providers are encouraged to also review the findings of the overall evaluation of the learning gain Pilot Projects for approaches which are most suited to their local contexts.

And finally the report on Higher Education Learning Gain Analysis (HELGA): says:

• “HELGA set out to explore whether administrative data could be used to create a proxy measure for learning gain. The project has experimented with different techniques, considered different outcomes that could be a proxy for learning gain and different source of data that could be used. Ultimately, the project has developed two methods for measuring value-added in higher education for a subset of the undergraduate population.
• The two measures have produced different results in their measure of value-added and there is no straightforward way of evaluating which is the most accurate.
• It should be noted that the ‘value-added’ measured is the difference between the ‘expected’ degree outcomes for students at an institution based on prior attainment (and other student and course characteristics in the case of the multilevel model) and the actual degree outcomes. The measure does not explain what this difference might be caused by. As mentioned in Section 2.3, there is concern about the comparability of degree classifications across institutions. This raises a question of the suitability of this value-added measure for comparing institutions.
• Neither methodology should be used further without additional sensitivity analyses and serious thought as to what is really being measured and whether it is fair to measure institutional performance in terms of value-added based on the restricted population used.
• At the outset of HELGA, it was known that it would never be possible to create a single measure of learning gain, encompassing all of the different elements that are understood to make up learning gain. It has necessarily focused on cognitive gain only, although some thought was given to using NSS metadata to measure non-cognitive learning gain, but this was unsuccessful. However, this does not mean that this could not be useful for measuring value-added, but it should be made clear that it should not be adopted to produce a single measure of learning gain”

Grade Inflation

And that brings us nicely to the next item…the OfS have published their latest review of degree classifications:

• The proportion of first-class honours degrees awarded has increased from 16 per cent to 29 per cent between 2010-11 and 2017-18.
• The OfS has used statistical modelling to account for factors including entrance qualifications and student characteristics which may influence attainment. When accounting for these factors, they find that 13.9 percentage points’ worth of first class degree attainment remains “unexplained”.
• In total, 94 per cent of the 148 universities and other higher education providers included in the analysis demonstrated a statistically significant unexplained increase in the proportion of first-class degrees awarded in 2017-18 compared to 2010-11.
• The report updates data from a report issued in December 2018. Since that report, universities have collectively announced plans to act together to curb grade inflation via the work of the UK Standing Committee for Quality Assessment.
• Commenting on the report, Susan Lapworth, director of competition and registration at the Office for Students, said:
  • ‘Worries about grade inflation threaten to devalue a university education in the eyes of employers and potential students. So it is essential we regain and maintain public confidence in the reliability of degree classifications.
  • ‘This data shows a further increase in both the rate of first-class degrees awarded, and the proportion of those awards. These increases cannot be fully explained by the factors we have taken into account in our analysis.
  • ‘The performance shown in the new data pre-dates our call for the sector to take action on grade inflation, so we would not expect to see the impact of such actions in today’s report.
  • ‘There are, though, positive signs that the higher education sector has begun to tackle this issue. We welcome the steps taken by the UK Standing Committee for Quality Assessment and the positive response from universities to a recent consultation on the steps universities should take to demonstrate that standards are secure. We recognise that change will take time, but it remains absolutely crucial that students, graduates and the general public can be assured that the value of a degree is maintained. That is why concerted, focused, and sector-wide action is so important.
  • ‘Following today’s publication, the OfS will be contacting those universities and providers with the most significant unexplained increases in degree classifications. We will ask them to provide further information to help us understand how they account for these increases. In seeking this additional information we recognise that there are factors that could explain the increases – for example improvements in learning and teaching – that we have not been able to measure in our analysis.
  • ‘Given the significant public scrutiny of degree standards we want to understand how universities have assured themselves that they have, and continue to, apply consistent standards. Doing so will help ensure that the degrees that students work so hard for continue to enjoy public confidence.’

The report and Excel versions of the data tables have been published.

The use of the word “unexplained” (again) is shocking given that it means “unexplained by prior attainment and social advantage”.  Inevitably this has been picked up in the media and by the Education Secretary.

BBC story here with a Damien Hinds comment:

•  “Education Secretary Damian Hinds warned against “unfair practices”.  Mr Hinds said that if universities were giving many more top degrees without a legitimate reason, it was unfair on those who had studied to the same standard in previous years.  “We owe it to the hard-working students and institutions who play by the rules to stamp out this unfair practice,” said the education secretary. “Today’s figures are disappointing and risk compromising the public trust in the high standards of our universities,” he said.”

Wonkhe point out the escalation of threat level here: “Back in December, he said “I am urging universities to tackle this serious issue and have asked the Office for Students to deal firmly with any institution found to be unreasonably inflating grades” – so this feels like threat inflation to us.”

The OfS seem to be totally unaware of the damage that their choice of language may be doing.  In a blog, Susan Lapworth, the Director of competition and registration at the OFS, says about the plan to follow up with universities (emphasis added);

“To ensure that all universities, colleges and other registered providers are playing their part in maintaining the standard of degrees, we are likely to write to those providers that held degree awarding powers in 2010-11 and where the data show the most significant increases in the percentage of first class degrees awarded between 2010-11 and 2017-18. We’re focusing on providers with:

• a statistically significant increase in the unexplained percentage of first class degrees awarded in a single year, or
• a statistically significant overall increase in the unexplained percentage of first class degrees awarded between 2010-11 and 2017-18.

We will ask them to provide further information to help us understand how they account for these increases. We want to understand, for example, whether a provider has made recent changes to the way it calculates degree classifications, or whether it can point to other evidence – such as investment in staffing, teaching, services or facilities – that would credibly account for the ‘unexplained’ increase. We are also interested in the steps governing bodies have taken to ensure that academic governance arrangements are adequate and effective.

In seeking this additional information, we are not implying that the trends we can see in the published data indicate any form of wrongdoing from these providers – we are trying to understand better the reasons for performance that will be subject to public scrutiny and so are focusing our attention on those providers with the biggest unexplained increases. Given the significant public scrutiny of degree standards we want to understand how providers have assured themselves that they continue to apply consistent standards.

Doing so is essential to maintaining public confidence in degrees.”

General Election?

There has been little to say on Brexit recently because of the speculation and posturing of the Conservative leadership race. The news is all about what the two candidates might actually do (rather than what they say they will do as some promises may turn out to be completely unachievable if the EU or Parliament don’t play ball). The Conservatives, despite bitter Brexit infighting, are keen to retain power and remain in Government, avoiding an election at all costs. However, there has been increasing talk of how a general election may now be inevitable. There is a good article in Politics Home House magazine which explains the election scenarios.

Admissions

UCAS released their analysis of all full time undergraduate applications (made by end June 2019) noting a new record as almost 4 in 10 young people apply to university. Overall the number of young applications has increased by 1%, an additional 2,600 people, (despite the 1.9% fall in the young UK population). Across the UK figures are:

• 39.5% of all 18 year olds in England submitted a UCAS application, up from 38.1% at the same point last year;
• in Northern Ireland, 46.9% of 18 year olds applied (down 0.7 percentage points)
• in Scotland, the 18 year old application rate is 32.7% (down 0.1 percentage point)
• in Wales, the application rate was 32.9% (up 0.2 percentage points) – a joint record result equalling the peak in 2016

International

• The volume of EU applicants rose by 1%, to 50,650.
• There were a record number of applicants from outside the EU – 81,340 students applied to study in the UK, an increase of 8%.
• China continued its rapid growth, with applicant numbers up 30% to 19,760 – this means that, for the first time, there are more applicants from China than Northern Ireland (18,520).

Disadvantage

For the first time, UCAS utilised the index of multiple deprivation measures to consider applications from disadvantaged communities.

• In England, the number of young people applying from the most deprived areas has increased 6% to 38,770, while applications from the least deprived areas have fallen.
• In Northern Ireland, all areas have seen a fall in applications, of between 2% and 7%.
• In Scotland, young applicants from the most deprived areas have grown by 3%, while all other areas have seen falls.
• In Wales, applicants from the most deprived areas remained at 1,390, with a mixed picture across different areas.

UCAS have also release a new interactive dashboard should you wish to interrogate the data further.

Clare Marchant, UCAS’ Chief Executive, said:

The global appeal of UK higher education has never been clearer, with record, demographic beating application rates in England and Wales, and the steep rise in international applications, especially from China.

Today’s analysis shows how attractive undergraduate study continues to be for young people, although university isn’t the only route on offer. Our survey insight shows that around a quarter of students are interested in apprenticeships as an alternative option.

Higher Technical Reform

The DfE has launched a consultation on Higher Technical qualifications and published a Written Ministerial Statement to accompany it. Key points within the statement:

1. A vision for Higher Technical Education to be a prestigious choice that delivers the skills employers need, encourages more students to continue studying after A levels or T levels and attracts workers of all ages looking to upskill and retrain.
2. The starting point for reform is to raise the prestige of Higher Technical Education and strengthen its value to employers by putting their needs and quality first. Improving quality now – to demonstrate the value of higher technical qualifications – will lead to increased uptake of Higher Technical Education in the future.
3. To do this we a new system is proposed to make clear which higher technical qualifications provide the skills that employers want. This will be delivered through the Institute for Apprenticeships and Technical Education signalling which qualifications deliver the knowledge, skills, and behaviours set out in employer-led national standards. This will help qualifications at this level command the confidence of students and employers alike.
4. Alongside this the Government intent to work with the Office for Students to demonstrate the quality of providers, so there is more high-quality provision delivered across higher and further education, including through our flagship employer-led National Colleges and Institutes of Technology.
5. The Government aims to make Higher Technical Education a positive and more popular choice by raising awareness and understanding of the new suite of Institute-approved qualifications in colleges and universities, and among potential students and employers.

This is very interesting in its own right, but also because of the direction of travel.

• 29 We will create a clear set of signals that will enable employers and learners to easily identify the best qualifications with national labour market relevance. We want to incentivise providers to gravitate towards approved qualifications rather than those that have not met the Institute’s quality requirements. This will tilt the playing field towards qualifications which have been identified by panels of employers as delivering the knowledge, skills and behaviours needed for an occupation.
• 30 Qualifications approved by the Institute would be clearly identified through a single name or kitemark. ….
• 31 Approved qualifications will therefore clearly stand out as being high-quality, labour market relevant, and having national currency. These benefits will enhance the offer and credibility of even those existing qualifications that have a relatively good level of employer awareness.
• In addition, we know it will be very important to ensure that HTE is properly funded. Funding will also form an important incentive for Awarding Bodies to submit their qualifications for approval and discourage provision of rival qualifications that have not been approved by the Institute. The Post-18 Review panel has recommended that only approved HTQs should be entitled to the same tuition fee support and teaching grant, and equivalent maintenance support, as level 6 qualifications. We want to ensure there are clear incentives to deliver reformed qualifications in the future, and we will consider this as part of the ongoing Spending Review and the government’s response to the Post-18 Review.
• We want to ensure that the very best providers are delivering our approved high-quality HTQs, whilst also considering how providers could specialise, or adjust their offer to address local skills needs. The Post-18 Review has recommended additional capital funding, for FE colleges, with a particular focus on growing specialist HTE provision in specific colleges. We will consider these recommendations, and develop our plans in more detail, as part of the Spending Review.
• We want the reformed higher technical offer to be the best it can be. We therefore propose taking the requirement to register with the OfS a step further, and to develop with the OfS an additional set of ongoing registration conditions specifically for higher technical provision (technical conditions). Providers would be required to meet these technical conditions, in addition to the general ongoing registration conditions that are applicable to all providers, to be able to deliver the approved HTQs with access to relevant student finance for courses leading to those HTQs, and to be eligible for any additional public funding.
• We expect this to be a rigorous process that would specifically assess and signal the quality of a provider’s higher technical education provision. Criteria for this would:
• Specifically indicate high-quality higher technical provision by expanding on the key elements already assessed by OfS as part of the registration process, such as:
  • Suitably qualified and experienced teachers with current, relevant occupational and industry experience and expertise, as well as high quality pedagogical skills. Leaders have the capacity and ability to ensure provision is sustainable and retains a clear focus on quality
  • Strong links with employer networks, thus ensuring the knowledge, skills and behaviours being delivered are valued by, and relevant to, employers who are engaged and investing in training; and
  • Learning environments that provide access to facilities and equipment that are reflective of the workplace, including industry-relevant, up-to-date equipment.
• Draw from the IoT assessment process, which uses a range of criteria including evidence of support for regional and national economic growth; employer engagement; relevance to occupational skills needs; and quality industry relevant teaching.
• 67 The panel has also recommended that additional support and capital funding should be provided to grow capacity for, and ensure, high-quality technical provision, and that quality indicators could identify how best to allocate that funding. This could mean that only providers meeting the technical conditions would be eligible for any higher technical capital or grant funding for their HTE provision. For example, this could be to support providers to expand their provision, specialise or tailor their offer to meet local skills needs, or enhance teaching facilities and equipment. As stated above, we will be considering these funding proposals and wider reforms as part of the Spending Review.

International staff

This came up in Parliamentary questions this week.

Q – Tom Brake (Carshalton and Wallington): To ask the Secretary of State for the Home Department, if he will make it his Department’s policy to exclude scientific research occupations from proposals in the immigration White Paper for a minimum salary threshold.

A – Caroline Nokes (Romsey and Southampton North):

• On 24 June 2019, the Government asked the independent Migration Advisory Committee (MAC) to consider the operation of salary thresholds in the future immigration system, including the impact of exemptions from minimum salary thresholds. The MAC is due to report by January 2020.
• We recognise the vital contribution that scientists make to the UK. In his spring statement, my Rt Hon Friend, the Chancellor of the Exchequer, confirmed that PhD level occupations would be exempt from the Tier 2 cap. Additionally, researchers applying for settlement are exempt from the rule which states that, there should be no absence from the UK for 180 days if the absence from the UK is for the purpose carrying out research. A number of research roles also appear on the Shortage Occupation List which also exempts them from the settlement salary threshold
• The Tier 1 (Exceptional Talent) route is also available for internationally recognised leaders and promising future leaders, including in the science and research sector.

The Minister speaks (part 4) – on R&D investment

Universities Minister, Chris Skidmore, made his fourth (and final) speech in his series on R&D investment focusing on how to bring about major increases in private R&D investment.

The previous speeches covered:

• Investing in people – ensuring that the best and brightest minds are working on the problems
• Working collaboratives – across international borders to tackle the greatest challenges facing society
• Enduring the funding and regulatory systems nurture new technologies and support risk taking

On private R&D investment he said:

• If we are to deliver on our 2.4% commitment, we must be prepared to go much further and much faster. Overall investment in R&D will need to more than double in nominal terms, from current level of £35 billion per year to around £75 billion per year by 2027.
• And at least two thirds of this investment will need to come from business. For many industries the need to invest in R&D is a pressing one – vital to competitive advantage. Businesses that thrive on the cutting edge are adept at making the most of basic research. They track down the brightest minds to seek answers to their most pressing problems. And they take the lead on experimental development and innovation, lifting new discoveries out of the lab and into people’s lives, driving economic and social prosperity.
• That isn’t to say that the government won’t play its part. In fact, we are doing more than ever.

The Minister goes on to set out the additional funding, including a real terms QR funding uplift, and collaborative assistance that has led to successful business gains, such as Jaguar Land Rover committing to produce it’s electric cars in the UK (previously the company had been shedding staff and moving assets due to Brexit complexities).

• Bold government policy has led to bold business decisions.
• But to get to 2.4% we need to see significantly more than this. To ramp up from today’s levels to 2027, it is projected we’ll need to see an additional £12 billion each year from business.

To respond to this challenge the minister announced 7 focal areas:

• Creating opportunity for the spark of creativity to arise through encouraging a vibrant and diverse research system with support for world-class, blue-skies research in universities and institutes… intellectual capital ultimately comes from people. So we need to invest confidently into postgraduate study, looking at all levels of our education system to getting more people into exciting R&D careers. This also means working tirelessly to put innovation and creativity at the centre of our education and training system.
• Turning new ideas into new businesses, products and processes – developing a culture of startups.
• Creating a scale-up culture with early-stage ventures able to access the funding needed to scale up
• Driving up demand as well as supply – incentivising business investment into R&D for established businesses. The  Minister states:

I see universities as ‘protagonists’, working with businesses to address problems where others cannot or dare not, and stimulating private investment.

Whether they are spinning out a company, licensing their IP, or undertaking contract or collaborative research with business, universities are remarkably skilled at identifying where they can have the greatest impact – locally, regionally, nationally and globally – and just getting on and doing it.

• Supporting innovation across the whole UK – playing to local and regional strengths.
• Focussing on the industrial strategies Grand Challenges

• Ensuring research and innovation is right at the heart of Global Britain – the UK to be a magnet for foreign direct investment and as attractive a destination as possible for major businesses looking to relocate or scale up their R&D operations. Government policies and funding streams should ruthlessly drive up our global attractiveness.
  • But it is also incumbent on us to be better at marketing ourselves overseas. An important part of this is being open to talent. It is something businesses tell me constantly: the UK urgently needs an immigration system that encourages those with the best ideas and the most innovative minds to come and work here.

• The evidence is unarguable: skilled immigration drives innovation. We mustn’t lose sight of this as we exit the EU. If we are to thrive, we need a genuinely international approach, a ‘freedom of talent’ approach that allows highly skilled people the flexibility to move and collaborate across borders.
• That is the purpose of our international education strategy, providing focus and clarity to our ongoing educational exports
• We cannot duck this challenge. Other nations are adopting aggressive approaches to R&D investment, driving significant increases in both public and private R&D performance. I’m not just talking about Israel and China – 2 obvious outliers. I’m also looking at Germany, who are ramping up public spending on R&D, committing to 3% annual increases every year for the next decade.
• The technological revolution will stretch horizontally across our economy – every company will become a tech company of sorts. 
• So to conclude, delivering on 2.4% requires focusing on all parts of the innovation lifecycle, and looking at our policies and investments through multiple lenses. And we must keep a ruthless focus on evidence, to inform the choices we make as a nation.
• This means we need to access the latest thinking on what works, and ensure we have the best and latest metrics so we know when we have succeeded. And it means listening to what businesses and academics are telling us, and acting accordingly.

Other points made:

• The UK has the second highest number of doctoral graduate in the EU (Germany has the highest).
• UK researchers are unusually collaborative in comparison with EU and G20 nations
• The UK has above-average level of people working in the fastest-growing sectors
• The UK’s field-weighted citations has beaten all other G7 countries since 2007 with 50% greater citation impact than the world average and 30% higher impact than the EU27. No other comparator nation has a larger proportion of its research among the most widely cited in the world.

Consultations

There are NINE new consultations and inquiries this week!  Click here to view the updated tracker. Email us on policy@bournemouth.ac.uk if you’d like to contribute to any of the current consultations.

Other news

Assistive Technology: The Student Loans Company plan to put out a tender for Assistive Technology Equipment and Training, results will be published during autumn 2019.

Industry partnership: Universities Minister, Chris Skidmore, announced UK pioneering technologies under development as part of 4 new partnerships between businesses and universities. The projects aim to help UK industry and academia lead the way in bringing new products to market that contribute to tackling the big generational challenges such as climate change and the needs of an ageing society. Skidmore stressed the importance of both government and industry contributing to the Industrial Strategy ambition of raising public and private investment in research and development to 2.4% of GDP by 2027. Projects include:

• Developing new materials that do not make noise underwater, led by BAE Systems with the University of Southampton, the University of Nottingham and Lloyd’s Register.
• Using AI and machine learning to speed up production of new medicines from vaccines to tablets in order to get them from the lab to the clinic faster, led by GlaxoSmithKline with the University of Strathclyde with University of Nottingham.
• Developing a new range of fully recyclable ultra-high strength aluminium alloys for the automotive industry, led by Constellium and Brunel University.
• Creating the next generation of household products using AI to pave the way for robots to complete advanced household tasks, led by Dyson and Imperial.

Student transitions:  Education Secretary Damian Hinds announced ‘Leapskills Workshops’, developed by student accommodation provider Unite Students which offer schools and colleges resources to teach Year 12 and 13 pupils about independent living, managing money and dealing with conflict. The sessions aim to act as a digital interactive masterclass to enhance how schools and colleges teach young people about what to expect and how to prepare for the leap of living away from home for the first time.

• Education Secretary Damian Hinds said:  For young people leaving school, starting the next chapter of their life should be a positive life-changing experience – but we know that many people struggle with the pressures of moving away from home and living independently for the first time. A huge part of education is preparing young people for adult life and it is right that we teach them what to expect for life after school, whether that’s at university, work or an apprenticeship.
• Unite Students CEO Richard Smith said: We believe that resilience is vital in young people and that given the right opportunities and experiences, young people can build resilience. The better prepared young people are for the transition to university, the easier they will find managing the highs and the lows often involved in this leap.

Apprenticeships: The DfE have published a summary document giving Apprenticeships and Traineeships figures. The data shows how higher level apprenticeships have boomed (68% growth since last year) with the main decline at the intermediate apprenticeship level. Apprenticeship starts from mature (19+) learners has increased by 13.8%.

Subscribe!

To subscribe to the weekly policy update simply email policy@bournemouth.ac.uk

JANE FORSTER                                            |                       SARAH CARTER

Policy Advisor                                                                     Policy & Public Affairs Officer

Follow: @PolicyBU on Twitter                   |                       policy@bournemouth.ac.uk