Category / Research Integritiy

SciVal’s Field weighted citation impact: Sample size matters!

There’s been a buzz on social media recently about Field weighted Citation Impact (FWCI) particularly around the recent leak from the University of Manchester that the FWCI is one of the measures suggested by which to assess academics most at risk of redundancy:

In his recent blog on The Bibliomagician Blog  (reposted here with permission) Iain Rowlands a Research Information & Intelligence Specialist at King’s College London and a member of the LIS-Bibliometrics committee questions the stability of the FWCI indicator for sets of fewer than 10,000 documents. Ian invites others to use his methodology to further test his theory…

SciVal’s field-weighted citation impact (FWCI) is an article-level metric that takes the form of a simple ratio: actual citations to a given output divided by the expected rate for outputs of similar age, subject and publication type.  FWCI has the dual merits of simplicity and ease of interpretation: a value of 2 indicates that an output has achieved twice the expected impact relative to the world literature.  It is a really useful addition to the benchmarking toolkit.

The trouble is that, typically, the distribution of citations to outputs is highly skewed, with most outputs achieving minimal impact at one end and a small number of extreme statistical outliers at the other.  Applying the arithmetic mean to data distributed like this, as does FWCI, is not ideal because the outliers can exert a strong leveraging effect, “inflating” the average for the whole set.  This effect is likely to be more marked the smaller the sample size.

I explored this effect in a simple experiment.  I downloaded SciVal FWCI values for 52,118 King’s College London papers published up until 2014.  I then calculated mean FWCI and 95% confidence (or stability) intervals for the whole sample using the bootstrapping[1] feature in SPSS.  Then I took progressively smaller random samples (99%, 98%, and so on to 1%, then 0.1%), recalculating mean FWCI and stability intervals each time.

The findings shows how mean FWCI becomes less stable as sample size decreases.  Highly cited outliers are relatively uncommon, but their chance inclusion or exclusion makes a big difference, especially as the number of outputs decreases.  In this experiment, FWCI values range across four orders of magnitude, from 0.03 to 398.28.

FWCI chart_black

What does this mean for interpreting FWCI, especially when benchmarking? The table below offers some guidance.  It shows typical stability intervals around FWCI at different scales.  The final column assumes that SciVal spits out a value of 2.20 and shows how that figure should be interpreted in terms of its stability.

FWCI Table

It’s pretty clear from this analysis that you need to know when it’s time to stop when you are drilling down in SciVal!  Another implication is that there is no sensible justification for quoting FWCI to two let alone three decimal places of precision.  I’ve kept the second decimal place above simply for purposes of demonstration.

I am well aware that the guidance above is based on data from just one institution, and may not travel well. If you would like to replicate this experiment using your own data, I’m happy to share my SPSS Syntax file.  It automates the whole thing, so you just have to load and go off on a short holiday! Just drop me an email.

Ian Rowlands is a Research Information & Intelligence Specialist at King’s College London and a member of the LIS-Bibliometrics committee.

ian.rowlands@kcl.ac.uk

Workshop: Building resilience in Research and Knowledge Exchange 27/4/17

Professor Heather Hartwell will be delivering a workshop on April 27th 2017 that will help participants gain insight into how it is possible to build resilience in the area of Research and Knowledge Exchange.

This session will explore how it may be possible to build resilience in the area of research and knowledge exchange, where rejection for funding and from publishers is common. The speaker will offer their views of how resilience can be built and how to overcome obstacles. There will be the opportunity for discussion around the topic.

For those interested in booking onto the course, please follow the link here.

If you would like further information about the workshop, please contact Ehren Milner (emilner@bournemouth.ac.uk)

 

HE Policy update w/e 31st March 2017

This week has of course been all about Brexit with the Article 50 notice formally served on Wednesday.   Bu published information for staff and students on Wednesday.

Then the White Paper setting out the plans for the Great Repeal Bill which will deal with arrangements to make UK law work once the UK has left the EU – a challenge given how much EU law has been incorporated into UK laws and regulations. The Bill itself has not been published, but the approach it will take is set out in the White Paper:

  • “In order to achieve a stable and smooth transition, the Government’s overall approach is to convert the body of existing EU law into domestic law, after which Parliament (and, where appropriate, the devolved legislatures) will be able to decide which elements of that law to keep, amend or repeal once we have left the EU. This ensures that, as a general rule, the same rules and laws will apply after we leave the EU as they did before….
  • The approach outlined in this White Paper is designed to give businesses, workers, investors and consumers the maximum possible certainty as we leave the EU: but it also needs to provide the flexibility necessary to respond to all eventualities of the negotiation process.
  • This is a separate process from that by which the Government will bring forward a motion on the final agreement to be voted on by both Houses of Parliament before it is concluded.
  • The Great Repeal Bill will not aim to make major changes to policy or establish new legal frameworks in the UK beyond those which are necessary to ensure the law continues to function properly from day one. Therefore, the Government will also introduce a number of further bills during the course of the next two years to ensure we are prepared for our withdrawal – and that Parliament has the fullest possible opportunity to scrutinise this legislation”.
  • The most controversial part of the proposal relates to powers to make secondary legislation. The White Paper says “This will enable corrections to be made to the laws that would otherwise no longer operate appropriately once we have left the EU, so that our legal system continues to function correctly outside the EU, and will also enable domestic law once we have left the EU to reflect the content of any withdrawal agreement under Article 50.”

So will there be any changes to UK law linked to Article 50 – there are some clues in the White Paper:

  • Customs bill
  • Immigration bill
  • And on consumer protection, the Government intends to bring forward a Green Paper this spring which will closely examine markets which are not working fairly for consumers

What the White Paper says won’t change:

  • Environmental protection
  • Worker’s rights and equalities
  • Participation in European Convention on Human Rights (ECHR), but not the EU Charter on Human Rights

Read more about the process for Brexit here.

Universities are not mentioned in the paper (except in the introduction where it is noted that the government acted fast to reassure applicants about fees) – there are on-going calls for the question of EU citizens in the UK to be settled fast, along with fee guarantees for students starting in 2018/19. UUK’s priorities for Brexit are set out in their short paper here and there are some interesting views written for the Universities All Party Parliamentary Group here.

Fake Research – UUK has a new blog on fake research following comments in the news about it. The blog helpfully brings together the latest reports and information in this area including the Research Councils UK guidance issued recently.

Higher Education and Research Bill – this will have its third reading in the House of Lords on Tuesday No more amendments have been added since the debate was postponed following the Westminster attacks. However, we know a little bit more about the government’s approach to the opposition and cross bench amendments made by the Lords. The bill will go back to House of Commons (to start its “ping-pong”) after Easter.

For example, of the 5 non-government amendments, one related to the requirement that universities should share information with local authorities to ensure that students are registered to vote. Jo Johnson has written a letter to HEFCE requesting that they do more to encourage institutions to support students to register to vote. They ask HEFCE to develop a best practice model and then encourage universities to use it. This amendment this therefore likely to be removed in the House of Commons

Another provided that the OfS could not approve a provider unless it has been validated for at least 4 years (as now) or has been approved by a Quality Assurance Committee as being full able to maintain the required standard for the duration of its authorisation and that it operates in the public interest and the interest of students. It seems unlikely that this will survive in the House of Commons:

  • A letter dated 22nd March from Jo Johnson sets out the government’s position that the student protection arrangements should cover this and that there will be a consultation on this as part of a Regulatory Framework consultation in the autumn of 2017.
  • A letter dated 8th March from Jo Johnson sets out why the government believes that the current validation arrangements do not work and explains why it should be easier for new providers to be authorised.

The other 4 amendments were:

  • Requiring UKRI to encourage international collaboration, not allowing students to be treated as long term migrants and not allowing more visa restriction on student or staff immigration more stringent than the day the act is passed – this is likely to be removed in the Commons pending the wider consultation on immigration policy which has been delayed since November
  • Removing the TEF clause and requiring the OfS to introduce instead a scheme to provide information about quality, which is approved by Parliament and which cannot be used to create a single composite ranking – This is likely to be removed in the Commons but it will be interesting to see if any concessions are made about the TEF. So far subject level TEF has been postponed for a year to allow for 2 years of pilot, but other changes may be forthcoming
  • A statement that the TEF (or its replacement) cannot be used to rank institutions as to the fees that they charge or the number of students they recruit, in the UK or overseas – this seems unlikely to survive – the student number/immigration issue will be dealt with as above, and the fee issue is so key to the government’s position on fees that it seems very unlikely to be dropped. There has been surprisingly little pick up on this issue – see the VC’s blog on this. There is a blog by Professor Mark Smith here.
  • The grounds in which institutions can appeal the revocation of their authorisation, so that instead of being on the grounds of an error of fact, a decision being wrong in law or unreasonable, it now just says that institutions can appeal on the grounds that “the decision was wrong” – this seems unlikely to survive – see the letter from Viscount Younger and Lord Young dated 20th

Apprenticeships – The Commons Select Committee on Education, Skills and the Economy have published a report on apprenticeships which raises a number of concerns about the focus on quantity over quality.

Social Mobility – Justine Greening gave a speech on 30th March on social mobility. She set out three priorities:

  • tackling geographic disadvantage
  • investing in long-term capacity in our system
  • making sure our education system as a whole really prepares young people and adults for career success

This speech refers to the plans for schools to work with universities but doesn’t give any more information (we are still waiting for the response to the consultation), technical education and widening access to universities.

Bringing FUSION to Nepal

FUSION abroad 2016We have written in many previous BU blogs about progress of our THET-funded project in southern Nepal (e.g. here AND here ). Today’s blog reflects on the use on BU’s unique FUSION approach in our project ‘Mental Health Training for Maternity Care Providers in Nepal‘.

DSC_0151Our BU-led project brings highly experienced health professionals, such as midwives, health visitors or mental health nurses, to Nepal to work as volunteer trainers. The training is aimed at community-based maternity care practitioners and addresses key mental health issues relevant to pregnancy and for new mothers and offers the required communication skills. These health professionals will bring their experience as health care providers as well as trainers in the field of mental health and maternity care/midwifery, mental ill-health prevention and health promotion. They volunteer for two to three weeks at a time to design and deliver training in southern Nepal.

logo THETThe Centre for Midwifery & Maternal Health (CMMPH) collaborates in this project with Liverpool John Moores University (LJMU), the Department of Health, and Physical & Population Education at Nepal’s oldest university Tribhuvan University’s (TU). The project is supported in the field by a local charity called Green Tara Nepal. Our project is part of the Health Partnership such as Nepal. HPS itself is funded by the UK Department for International Development and managed by THET (Tropical and Health Education Trust).

Fusion Diagram Our maternal mental health project is a good example of BU’s FUSION approach as it combines EDUCATION (through the training of Auxiliary Nurse-Midwives in Nepal) by UK volunteers (representing PRACTICE) through an intervention which is properly evaluated (representing RESEARCH) is a perfect example of BU’s FUSION in action. Moreover, the project will be partly evaluated by FHSS’s Preeti Mahato as part of her PhD thesis research. This PhD project is supervised by Dr. Catherine Angell (CEL & CMMPH), BU Visiting Professor Padam Simkhada (based at LJMU) and CMMPH’s Prof. Edwin van Teijlingen.BU’s focus on the FUSION of research, education and professional practice is a unique variant of the way UK universities (and many abroad) blend academic teaching, research and scholarship. FUSION is a key concept derived from BU’s strategic Vision & Values).

 

Prof. Edwin van Teijlingen

CMMPH

Why editorials?

Zika editorial 2016BU academics are editors on a wide range of scientific journals.  As editors we often write editorials for academic journals which have a number of specific functions.  It is a key means of communication between the editor(s) and the journal’s readership.  It is also vehicle to highlight topical academic and political issues related to the journal and the discipline(s) it represents. JAM June 2016 editorial

Earlier this week the latest issue of the Journal of Asian Midwives came out with an editorial which is an illustration of the first point giving information to the readers [1].  The topics addressed in this editorial included the announcement that this new journal was now indexed in the CINAHL Database, a recent major international conference in the field and a call for the forthcoming 2017 ICM (Internation Confederation of Midwives) tri-annual conference.  Today saw the publication of an editorial on the Zika virus and its potential impact in Nepal in the journal Medical Science [2].   This guest editorial co-written by BU’s Visiting Faculties Dr. Brijesh Sathian and Prof. Padam Simkhada with Prof. Edwin van Teijlingen (Centre for Midwifery, Maternal & Perinatal Health) calls for action in Nepal.  A country where malaria is endemic. The Zika virus uses mosquitoes like the ones spreading Dengue fever and malaria.  Zika is a virus we do not wish to see spreading in countries where malaria is already rife.  The editorial warns that precautionary measures are needed to prevent a Zika outbreak as the spread of the virus to the country seems inevitable, the only uncertainty is when it will be arriving.

Both journals are Open Access which means these editorials can be read by anybody with internet access free of charge.

References:

  1. Jan, R., van Teijlingen, E. (2016) Editorial JAM June 2016, Journal of Asian Midwives 3(1):1. http://ecommons.aku.edu/jam/vol3/iss1/1/
  2. van Teijlingen, E., Sathian, B., & Simkhada, P. (2016). Zika & Nepal: a far greater risk for its population than to individuals. Medical Science 4(2): 312-313. http://www.pubmedhouse.com/journals/ms/articles/1064/PMHID1064.pdf

 

Announcement BU Humanisation Conference 2016

BU Humanisation Conference     21st June 2016

Venue: Room EB708, Executive Business Centre, 89 Holdenhurst Road, BH8 8EB

 

Please find the Programme for the Humanisation conference on the 21st June 2016 attached.

Please feel free to pass the information on to others internal and external to the university (academic and practice) who you feel may be interested

The conference is being run at no cost and so you need to make your own arrangements for lunch.  Let Dr. Caroline Ellis-Hill  ( cehill@bournemouth.ac.uk ) know by the 15th June if you wish to attend .

If you only want to attend for part of the day, please state which part of the day you’d like to attend.

 

9.30 Registration  
10.00 Dr Caroline Ellis-Hill Welcome
10.10 Anne Quinney Humanisation of the BU Generic Student Assessment Criteria.
10.30 Dr Sean Beer Perceptions of the authenticity of food: a study of residents in Dorset (UK)
10.50 Prof Ann Hemingway Innovative routes to Wellbeing: Equine Assisted interventions
11.10 Coffee  
11.30 Jane Fry Sharing human concerns: utilising an embodied interpretative approach to convey findings from a descriptive phenomenological study
11.50 Dr Carole Pound Humanising care: translating theory into practice in stroke care
12.10 Rutherford and Dr. Emer Forde The Rutherford Introspective Photography: Promoting self-reflection and wellbeing of GP trainees through photography.
12.30 Free time   Please see information about local venues for lunch
2.00 Dr Vanessa Heaslip How phenomenology enables insight into the Human lives of Gypsy Roma Travellers’
2.20 Mevalyn Cross Experiencing the Humanisation Framework together
2.40 Dr Jan Mosja Chaplaincy at the bedside. Learning from Buddhist chaplains and their contributions to the humanisation of health care.
3.00 Sally Lee Humanising and the Care Act well-being principle
3.20 Dr Mary Grant and Dr Catherine Lamont Robinson HeART of Stroke: feasibility study of an Art & Health intervention following a stroke
3.40 Thanks, Tea and Close  

 

The Concordat to Support Research Integrity

The Concordat to Support Research Integrity provides a comprehensive national framework for good research conduct and its governance. It outlines five commitments which the University must be compliant with as recipients of HEFCE funding.

HEFCE’s letter to all the heads of HEFCE-funded higher education institutions explains the requirement for institutions to comply with the Concordat to Support Research Integrity. It sets out HEFCE’s view of compliance with the Concordat, and provides advice for institutions on finding further information. The five commitments are:

  1. Maintain the highest standards of rigour and integrity in all aspects of research;
  2. Ensure that research is conducted according to appropriate ethical, legal and professional frameworks, obligations and standards;
  3. Support a research environment that is underpinned by a culture of integrity and based on good governance, best practice and support for the development of researchers;
  4. Use transparent, robust and fair processes to deal with allegations of research misconduct should they arise;
  5. Work together to strengthen the integrity of research and to review process regularly and openly.

Each commitment outlines the responsibilities of the researcher, the employers of researchers and the funders of research.

Bournemouth University is committed to ensuring that the research conducted at the Univeristy follows appropriate ethical, legal and professional frameworks and obligations.

A link to the Concordat can be found here:

http://www.universitiesuk.ac.uk/highereducation/Documents/2012/TheConcordatToSupportResearchIntegrity.pdf

Seminar, Prof Edwin van Teijlingen, ‘Maternal Mortality in Nepal’, Wed 20th April, Royal London House, R303, 13:00-13:50.

Maternal Mortality in Nepal
Abstract: The session links various social and political factors that affect maternal mortality. Women dying in pregnancy and childbirth is very much a problem of and in low-income countries. This talk focuses on Nepal, one of the poorer countries of the world, to highlight a range of maternal health issues and wider influencing factors including globalisation and the influence of global organisations such as the World Health Organisation.

For further information regarding the Social Science seminar series, get in touch with Dr Mastoureh Fathi (mfathi@boutnemouth.ac.uk).

Pritchard & Harding paper cited in top journal, but there’s a ‘but…’

cloudAs an ECR I am delighted to see that a research paper that Prof. Pritchard and myself wrote in 2014 has been cited in one of the most well regarded journals in the field.

Our paper on the occupational backgrounds of Non-executive directors at NHS acute trusts, published in the Journal for the Royal Society of Medicine Open, was also the subject of an article in the now defunct Independent  newspaper and a post on this blog in May 2014.

Last year, in 2015, it was cited by a paper published in the Journal for Health Services Research and Policy. I won’t name which edition or paper because there is a ‘but…’, and it concerns the carelessness of the authors who cited our work.

There is a ‘but…’ because the authors got my name wrong – both in the in text citation and in the bibliography. The good news is that it still links on citation tracking systems (such as the function on Google Scholar) as a paper that I co-wrote. Yet as an ECR, who is trying to make his way in the ‘publish or perish’ world of academia, I can’t help but feel a bit frustrated. Here’s my name in a top journal, but it’s incorrect.

So I took action, I emailed the editors. To their credit I got a response within minutes, with an apology for the carelessness of the authors and that contact with the publisher had been initiated to see if it could be corrected. Yet, due to the inflexibility of doi, apparently this is unlikely.

This then got me thinking about my first publication. I have to admit I did not check the final galley proof thoroughly enough. Indeed, when it was published, it became apparent that I had not corrected some basic incorrect spelling of names in the bibliography. In other words, some very respected authors’ names were wrong! I can happily report that this was corrected, and no offence caused (I hope!).

But the lesson here – check final galley proofs. If you cite an article, I think the very least you can do, out of respect for colleagues, is to get the authors name right. I have made this mistake, and so have authors who have cited me, so it would scream hypocrisy if I was too mad! But it does show that it might be a relatively common problem, so again – check final galley proofs!

However, once the relative pain bypassed, one our papers has still been cited in a top journal – and that is very satisfying indeed.