Category / Research Ethics

SciVal’s Field weighted citation impact: Sample size matters!

There’s been a buzz on social media recently about Field weighted Citation Impact (FWCI) particularly around the recent leak from the University of Manchester that the FWCI is one of the measures suggested by which to assess academics most at risk of redundancy:

In his recent blog on The Bibliomagician Blog  (reposted here with permission) Iain Rowlands a Research Information & Intelligence Specialist at King’s College London and a member of the LIS-Bibliometrics committee questions the stability of the FWCI indicator for sets of fewer than 10,000 documents. Ian invites others to use his methodology to further test his theory…

SciVal’s field-weighted citation impact (FWCI) is an article-level metric that takes the form of a simple ratio: actual citations to a given output divided by the expected rate for outputs of similar age, subject and publication type.  FWCI has the dual merits of simplicity and ease of interpretation: a value of 2 indicates that an output has achieved twice the expected impact relative to the world literature.  It is a really useful addition to the benchmarking toolkit.

The trouble is that, typically, the distribution of citations to outputs is highly skewed, with most outputs achieving minimal impact at one end and a small number of extreme statistical outliers at the other.  Applying the arithmetic mean to data distributed like this, as does FWCI, is not ideal because the outliers can exert a strong leveraging effect, “inflating” the average for the whole set.  This effect is likely to be more marked the smaller the sample size.

I explored this effect in a simple experiment.  I downloaded SciVal FWCI values for 52,118 King’s College London papers published up until 2014.  I then calculated mean FWCI and 95% confidence (or stability) intervals for the whole sample using the bootstrapping[1] feature in SPSS.  Then I took progressively smaller random samples (99%, 98%, and so on to 1%, then 0.1%), recalculating mean FWCI and stability intervals each time.

The findings shows how mean FWCI becomes less stable as sample size decreases.  Highly cited outliers are relatively uncommon, but their chance inclusion or exclusion makes a big difference, especially as the number of outputs decreases.  In this experiment, FWCI values range across four orders of magnitude, from 0.03 to 398.28.

FWCI chart_black

What does this mean for interpreting FWCI, especially when benchmarking? The table below offers some guidance.  It shows typical stability intervals around FWCI at different scales.  The final column assumes that SciVal spits out a value of 2.20 and shows how that figure should be interpreted in terms of its stability.

FWCI Table

It’s pretty clear from this analysis that you need to know when it’s time to stop when you are drilling down in SciVal!  Another implication is that there is no sensible justification for quoting FWCI to two let alone three decimal places of precision.  I’ve kept the second decimal place above simply for purposes of demonstration.

I am well aware that the guidance above is based on data from just one institution, and may not travel well. If you would like to replicate this experiment using your own data, I’m happy to share my SPSS Syntax file.  It automates the whole thing, so you just have to load and go off on a short holiday! Just drop me an email.

Ian Rowlands is a Research Information & Intelligence Specialist at King’s College London and a member of the LIS-Bibliometrics committee.

ian.rowlands@kcl.ac.uk

RCUK Policy and Guidelines on the Governance of Good Research Conduct

The RCUK Policy and Guidelines on the Governance of Good Research Conduct aims to help researchers and research organisations to manage their research to the highest standards, and provides guidance on the reporting and investigation of unacceptable research conduct.

The guide has been updated from 1 April 2017.  The updates include the need to notify the relevant research council of an allegation of research misconduct at the stage that it is decided to undertake an informal inquiry; not, as previously, at the (later) stage of deciding to undertake a formal investigation.  Please see the link above for the full changes.

*Book now* Research Application training- Spaces still available on the RKE Development Framework Pre-Award Pathway

The research and knowledge exchange (RKE) development framework offers a range of opportunities for academics at all career stages to develop their skills, knowledge and capabilities in relation to research and knowledge exchange. The pre-award pathway offers all of the starting information required by academics and researchers at BU to undertake research bidding.

Research Ethics at BU

All research being conducted at BU falls under the Ethics and Governance policies at BU. This session will offer Academics and Researchers an understanding of the Ethics procedures and Research Governance policies at BU.

10 April 2017 10.00 – 11.00 Lansdowne Campus

Getting started on applying for research funding

This session will explore how best to adapt research in response to the changing external environment. The workshop will provide information on the best routes to funding based upon career stages and also introduce how RKEO can help.

Thursday 13 April 2017 09.00 – 10.00 Lansdowne Campus

Pre-award finance

This session will introduce researchers to Full Economic Costs (fEC), transparant approaches to costing (TRAC) and the BU Financial Regulations. Guidance will be offered on how to cost projects in a way that funders will find acceptable. Training will be provided on producing the ‘Justificaton of Resources’ document required by many funders.

Thursday 13 April 2017 10.30 – 11.30 Lansdowne Campus

BU processes for applying for funding

This workshop will provide a short introduction/refresher on how to apply for external funding at BU. The latest update on the policies and processes will be introduced as part of this short session.

By the end of the session you will be familiar with the processes required to apply for funding at BU.

Thursday 13 April 2017 13.30 – 14.30 Lansdowne Campus

Quality approvals at BU

This course is aimed at those who are, or wish to be, a designated Faculty and UET Activity Quality/Peer reviewers. This session will provide an introduction/refresher of academic review policies at BU.

Thursday 13 April 2017 15.00 – 16.00 Lansdowne Campus

HRA Approval for NHS Research

HRA Approval is the new process for the NHS in England that simplifies the approvals process for research, making it easier for research studies to be set up. It replaces the need for local checks of legal compliance and related matters by each participating organisation in England. This allows participating organisations to focus their resources on assessing, arranging and confirming their capacity and capability to deliver the study.

Laura Purandare, Research Monitor RBCH, has kindly agreed to run a seminar on 4th May at 2pm in BG14 to explain the changes.

The session will cover:

  • What HRA approval is
  • The implementation of changes
  • The difference it proposes to make to health research in England
  • What it means for our researchers
  • Key resources

The session will last approximately an hour, and Laura will be available for questions following the session. We hope to see you there.

Seminar, Prof Edwin van Teijlingen, ‘Maternal Mortality in Nepal’, Wed 20th April, Royal London House, R303, 13:00-13:50.

Maternal Mortality in Nepal
Abstract: The session links various social and political factors that affect maternal mortality. Women dying in pregnancy and childbirth is very much a problem of and in low-income countries. This talk focuses on Nepal, one of the poorer countries of the world, to highlight a range of maternal health issues and wider influencing factors including globalisation and the influence of global organisations such as the World Health Organisation.

For further information regarding the Social Science seminar series, get in touch with Dr Mastoureh Fathi (mfathi@boutnemouth.ac.uk).

HRA Approval for NHS Research

HRA Approval is the new process for the NHS in England that simplifies the approvals process for research, making it easier for research studies to be set up. It replaces the need for local checks of legal compliance and related matters by each participating organisation in England. This allows participating organisations to focus their resources on assessing, arranging and confirming their capacity and capability to deliver the study.

Laura Purandare, Research Monitor RBCH, has kindly agreed to run a seminar on 4th May at 2pm in BG14 to explain the changes.

The session will cover:

  • What HRA approval is
  • The implementation of changes
  • The difference it proposes to make to health research in England
  • What it means for our researchers
  • Key resources

The session will last approximately an hour, and Laura will be available for questions following the session. We hope to see you there.

Talk by Dr Stephanie Schwandner-Sievers cancelled

Unfortunately we are cancelling the talk: Ethnographies of Memory – the cultural reproduction of militancy in Kosovo by Dr Stephanie Schwandner-Sievers. We apologise for the late notice. This is because the Women Academic Network talk has been rescheduled this morning for the same time slot (see below). Dr Schwandner-Sievers will give her talk at a later date to avoid some people having to make a choice.

Polly Trenow (Fawcett Society)
‘Feminism in practice – does activism really work?’
Wednesday 18th November
TAG22
2-4pm (with networking 3-4pm)

For any questions, please get in touch with Dr Masi Fathi (mfathi@bournemouth.ac.uk)

Open Access publishing does not have to be expensive!

Nepal J Epid Open AccessAs it is Open Access Week I would like to clarify one of the Open Access publishing myths.  One of the common replies I receive from academics colleagues when raising Open Access publishing is that it is (too) expensive. This is, of course, true for many academic journals, but not all are expensive.  Some don’t even charge a processing fee at all.  Infamously, The Lancet Global Health charges an article processing fee of US $4750 upon acceptance of submitted research articles.  More moderately priced scientific journals still charge anything up to about £1,500 per article.

Open-Access-logoAcademic publishing has been big business for decades, and Open Access has rapidly become part of that business.  While traditional book and magazine publishers struggle to stay afloat, research publishing houses have typical profit margins of nearly 40%, according CBCNEWS who quote Vincent Larivière from the University of Montreal’s School of Library & Information Science.

At the same time we see a sharp increase in so-called Predatory Publishers who have set up business for the sole reason to make money from Open Access publishing.  They have not established or taken over academic journal for the greater good of the discipline or the dissemination of research findings to the widest possible audience.  Unscrupulous publishers jump on the Open-Access bandwagon BU librarian Jean Harris recently shared an interesting article about Predatory Publishers (click here to read this!).

J Asian MidwHowever, there are other format of Open Access. One of our more recent papers on research ethics was published in the Nepal Journal of Epidemiology which is an online Open Access journal that does not charge authors for publishing!  Also the Journal of Asian Midwives, where FHSS PhD student Preeti Mahato recently had her article accepted, is hosted in Pakistan by Aga Khan University through its institutional repository eCommons.  Publishing in this Open Access online journal is also free of charge.  In other words, Open Access publishing does not have to be expensive!

 

Prof. Edwin van Teijlingen

CMMPH

 

Congratulations to CMMPH Professor Edwin van Teijlingen on his publication !

Congratulations to CMMPH Professor Edwin van Teijlingen on his latest publication about why researchers do not always seek ethical permission for health research conducted in low income countries. The authors in this paper have identified and explained five possible reasons; a) approval not needed: b) not familiar with the ethics committee: c) applying the wrethicsong committee; d) resource constraints; and e) assumption that non-clinical research are exempted, which are of course overlap and interact each other, for not applying ethical approval in low income countries. They have also provided examples of ethical approval taken from other countries than the host countries and further go on to stress that junior researchers and students should be encouraged to be familiar with research ethical approval. In their paper, they encourage journal editors and peer reviewers to ensure ethical approval beinProfessor Edwin and Professor Padamg granted for manuscripts based on empirical studies. This paper was co-authored by BU visiting faculty Professor Padam Simkhada and recently published in  Nepal Journal of Epidemiology. The paper is freely available through the journal’s website http://nepjol.info/index.php/NJE/issue/view/919

Reference:

van Teijlingen E, Simkhada P. Failure to apply for ethical approval for health studies in low-income countries. Nepal J Epidemiol. 2015;5(3); 511-515

 

Pramod R Regmi, PhD

Post Doctoral Research Fellow, Faculty of Health and Social Science