Category / writing

SciVal’s Field weighted citation impact: Sample size matters!

Guidance, Publishing, Research assessment, Research Ethics, Research Integritiy, writing

There’s been a buzz on social media recently about Field weighted Citation Impact (FWCI) particularly around the recent leak from the University of Manchester that the FWCI is one of the measures suggested by which to assess academics most at risk of redundancy:

In his recent blog on The Bibliomagician Blog  (reposted here with permission) Iain Rowlands a Research Information & Intelligence Specialist at King’s College London and a member of the LIS-Bibliometrics committee questions the stability of the FWCI indicator for sets of fewer than 10,000 documents. Ian invites others to use his methodology to further test his theory…

SciVal’s field-weighted citation impact (FWCI) is an article-level metric that takes the form of a simple ratio: actual citations to a given output divided by the expected rate for outputs of similar age, subject and publication type.  FWCI has the dual merits of simplicity and ease of interpretation: a value of 2 indicates that an output has achieved twice the expected impact relative to the world literature.  It is a really useful addition to the benchmarking toolkit.

The trouble is that, typically, the distribution of citations to outputs is highly skewed, with most outputs achieving minimal impact at one end and a small number of extreme statistical outliers at the other.  Applying the arithmetic mean to data distributed like this, as does FWCI, is not ideal because the outliers can exert a strong leveraging effect, “inflating” the average for the whole set.  This effect is likely to be more marked the smaller the sample size.

I explored this effect in a simple experiment.  I downloaded SciVal FWCI values for 52,118 King’s College London papers published up until 2014.  I then calculated mean FWCI and 95% confidence (or stability) intervals for the whole sample using the bootstrapping[1] feature in SPSS.  Then I took progressively smaller random samples (99%, 98%, and so on to 1%, then 0.1%), recalculating mean FWCI and stability intervals each time.

The findings shows how mean FWCI becomes less stable as sample size decreases.  Highly cited outliers are relatively uncommon, but their chance inclusion or exclusion makes a big difference, especially as the number of outputs decreases.  In this experiment, FWCI values range across four orders of magnitude, from 0.03 to 398.28.

FWCI chart_black

What does this mean for interpreting FWCI, especially when benchmarking? The table below offers some guidance.  It shows typical stability intervals around FWCI at different scales.  The final column assumes that SciVal spits out a value of 2.20 and shows how that figure should be interpreted in terms of its stability.

FWCI Table

It’s pretty clear from this analysis that you need to know when it’s time to stop when you are drilling down in SciVal!  Another implication is that there is no sensible justification for quoting FWCI to two let alone three decimal places of precision.  I’ve kept the second decimal place above simply for purposes of demonstration.

I am well aware that the guidance above is based on data from just one institution, and may not travel well. If you would like to replicate this experiment using your own data, I’m happy to share my SPSS Syntax file.  It automates the whole thing, so you just have to load and go off on a short holiday! Just drop me an email.

Ian Rowlands is a Research Information & Intelligence Specialist at King’s College London and a member of the LIS-Bibliometrics committee.

ian.rowlands@kcl.ac.uk

Writing Days – Book On!

Publishing, writing

As part of the Writing Academy, a series of writing days have been organised to help support BU authors work on their publications by providing some dedicated time and space, away from everyday distractions.

The days will have a collaborative focus on productive writing with other BU authors, the RKEO team will also be on hand to provide authors with help and guidance on all areas of the publication process.

Writing Days have been scheduled on the below dates:

  • Tuesday 9th May
  • Thursday 25th May
  • Friday 9th June
  • Monday 19th June
  • Tuesday 20th June
  • Wednesday 5th July
  • Thursday 27th July

Spaces are limited so please only book on if you are able to commit to attending for the whole day.

Click here to book on!

Free Peer Review Workshop for Early Career Researchers

Knowledge Exchange, open access, writing

Find out about peer review.

Debate challenges to the system.

Discuss the role of peer review for scientists and the public.

 

Friday 12th May, 2pm– 6pm

Workshop to be held at Informa’s Offices, 5 Howick Place, London

 

Peer Review: The nuts and bolts is a free half-day workshop for early career researchers and will explore how peer review works, how to get involved, the challenges to the system, and the role of peer review in helping the public to evaluate research claims.

 

Should peer review detect plagiarism, bias or fraud? What does peer review do for science and what does the scientific community want it to do for them? Should reviewers remain anonymous? Does it illuminate good ideas or shut them down?

 

To apply to attend this workshop, please fill out the application form by 9am on Tuesday 25 April: http://bit.ly/2mCFsyr

 

For more details, get in touch with Joanne Thomas jthomas@senseaboutscience.org.

More information: http://senseaboutscience.org/activities/peer-review-workshop/

Need tips on developing a publication strategy?

BRIAN, Guidance, open access, Publishing, writing

Then come along to one of the Writing Academy’s “My publication story so far…” lunchbyte sessions.

The first of 2017, is happening today at midday led by Prof. Matthew Bennett.

Matthew Bennett will be talking about his personal publishing experience, his approaches to research and writing, his tips on developing a publication strategy and working with co-authors, reviewers and editors. He will talk about all types of publishing from journal articles, to books via edited compilations. Drawing on personal experience publishing in Nature, he will also focus on how you target high impact journals.

Click here to book on!

Future sessions:

Prof. Tim Rees – Wednesday 24th May, 12-1.30pm

Prof. Sara Ashencaen Crabtree – Wednesday 28th June, 12-1.30pm

Click here to book on!

New midwifery paper by Dr. Jenny Hall

international, Publishing, writing

Congratulations to Dr. Jenny Hall in the Centre for Midwifery, Maternal & Perinatal Health (CMMPH) on the publication of her paper ‘Spiritual aspects of living with infertility: synthesis of qualitative studies’. [1]  Dr. Hall co-authored this paper in the Journal of Clinical Nursing with colleagues from Ireland and Portugal.

This international team conducted review and synthesis of qualitative research to seek a deeper understanding of the spiritual aspects of patients’ experiences of infertility.  They concluded that infertile couples’ experiences of infertility may offer an opportunity for spiritual care particularly related to the assessment of spiritual needs and the promotion of spiritual coping strategies. Moreover, effective holistic care should support couples in overcoming and finding meaning in this life and health condition.

Prof. Edwin van Teijlingen

CMMPH

Prof. Tim Rees – My publication story so far…

Guidance, open access, Publishing, writing

On Wednesday 24th May, the Writing Academy will be hosting a Lunchbyte session with Tim Rees. During the session Tim will talk about his personal publishing experience, his approaches to research and writing, his tips on developing a publication strategy and working with co-authors, reviewers and editors. He will talk about all types of publishing drawing on personal experience.

Aims:

  • Developing a Publication Strategy
  • Dealing with Co-Editors, Reviewers & Editors
  • Targeting high impact Journals

Click here to book on!

Dedicated Time and Space to Write…

BRIAN, Guidance, Publishing, writing

As part of the Writing Academy, a series of writing days have been organised to help support BU authors work on their publications by providing some dedicated time and space, away from everyday distractions.

The days will have a collaborative focus on productive writing with other BU authors, the RKEO team will also be on hand to provide authors with help and guidance on all areas of the publication process.

Writing Days have been scheduled on the below dates:

  • Tuesday 9th May
  • Thursday 25th May
  • Friday 9th June
  • Monday 19th June
  • Tuesday 20th June
  • Wednesday 5th July
  • Thursday 27th July

Spaces are limited so please only book on if you are able to commit to attending for the whole day.

Click here to book on!

Prof. Matthew Bennett – My publication story so far…

BRIAN, Guidance, Publishing, writing

writingOn Wednesday 29th March, the Writing Academy will be hosting a Lunchbyte session with Matthew Bennett. During the session Matthew will talk about his personal publishing experience, his approaches to research and writing, his tips on developing a publication strategy and working with co-authors, reviewers and editors. He will talk about all types of publishing from journal articles, to books via edited compilations. Drawing on personal experience, he will also focus on how you target high impact journals.

Aims:

  • Developing a Publication Strategy
  • Dealing with Co-Editors, Reviewers & Editors
  • Targeting high impact Journal

Click here to book on!

Writing Days

Publishing, Research assessment, writing

Imagine… dedicated time and space to write away from every day distractions.

Sounds great? Then you’ll be glad to hear that as part of the Writing Academy we are busy scheduling a number of Writing Days for BU authors to book on to.

Watch. This. Space.

New publication: vital signs obstetric charts

PG research, Publishing, student research, writing

Congratulations on the Faculty of Health & Social Sciences team which had its paper ‘Vital signs and other observations used to detect deterioration in pregnant women: an analysis of vital sign charts in consultant-led UK maternity units’ accepted by the International Journal of Obstetric Anesthesia (published by Elsevier). 
The paper compares: (i) vital sign values used to define physiological normality; (ii) symptoms and signs used to escalate care; (iii) 24 type of chart used; and (iv) presence of explicit instructions for escalating care. The authors conclude that the wide range of ‘normal’ vital sign values in different systems used in the UK and the Channel Islands suggests a lack of equity in the processes for detecting deterioration and escalating care in hospitalised pregnant and postnatal women. Agreement regarding ‘normal’ vital sign ranges is urgently required and would assist the development of a standardised obstetric early warning system and chart. The lead author of this new paper is FHSS Visiting Professor Gary Smith, his co-authors include FHSS staff Vanora Hundley, Lisa Gale_Andrews and Edwin van Teijlingen as well as three BU Visiting Faculty: Debra Bick (King’s College London), Mike Wee (Poole Hospital NHS Foundation Trust) and Richard Isaacs (University Hospital Southampton).